Information processing apparatus and non-transitory computer readable medium

ABSTRACT

An information processing apparatus includes a processor configured to acquire biological information of plural users determined to be within a predetermined area and operate an apparatus based on the biological information of the plural users.

RELATED APPLICATIONS

This application is a Continuation-in-Part (CIP) of U.S. patentapplication Ser. No. 17/131,683 filed on Dec. 22, 2020, which claims thebenefit of priority of Japanese Patent Application No. 2020-166688 filedon Oct. 1, 2020. The contents of the above applications are allincorporated by reference as if fully set forth herein in theirentirety.

FIELD AND BACKGROUND OF THE INVENTION (i) Technical Field

The present disclosure relates to an information processing apparatusand a non-transitory computer readable medium.

(ii) Related Art

It is possible that operation targets are operated using biologicalinformation such as brain waves.

In Japanese Unexamined Patent Application Publication No. 2014-071825, abrain-machine interface (BMI) apparatus that employs steady statevisually evoked potential (SSVEP) is described.

In Japanese Unexamined Patent Application Publication No. 2015-211705, aBMI motion assistance apparatus that detects a brain wave biologicalsignal from a user's brain waves, that detects a surface myoelectricpotential signal from the user's surface myoelectric potential, and thatobtains a control signal on the basis of these biological signals isdescribed.

In Japanese Unexamined Patent Application Publication No. 2015-102650,an imaging control apparatus that stores in advance brain waveconditions, which are conditions of brain waves caused when certainphysical motions are imagined, that obtains brain wave information, andthat, if brain waves included in the obtained brain wave informationsatisfy one of the stored brain wave conditions, outputs an imagingcondition, which is a condition at a time when an imaging apparatuscaptures an image of a subject, to the imaging apparatus is described.

In Japanese Unexamined Patent Application Publication No. 2002-166050, amethod for matching a brain wave signal and a person's intention isdescribed.

SUMMARY OF THE INVENTION

Aspects of non-limiting embodiments of the present disclosure relate toan operation of an operation target based on biological informationregarding plural users.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided aninformation processing apparatus including a processor configured toacquire biological information of plural users determined to be within apredetermined area and operate an apparatus based on the biologicalinformation of the plural users.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

An exemplary embodiment of the present disclosure will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram illustrating the configuration of aninformation processing system according to an exemplary embodiment;

FIG. 2 is a block diagram illustrating the hardware configuration of aninformation processing apparatus;

FIG. 3 is a block diagram illustrating the hardware configuration of aterminal apparatus;

FIG. 4 is a diagram illustrating a management table;

FIG. 5 is a diagram illustrating the management table;

FIG. 6 is a diagram illustrating the management table;

FIG. 7 is a diagram illustrating the management table;

FIG. 8 is a diagram schematically illustrating persons and a device in acertain place;

FIG. 9 is a diagram schematically illustrating the persons and thedevice in the certain place;

FIG. 10 is a diagram illustrating a screen;

FIG. 11 is a diagram illustrating the screen;

FIG. 12 is a diagram illustrating the screen;

FIG. 13 is a diagram illustrating the screen;

FIG. 14 is a diagram illustrating another screen;

FIG. 15 is a diagram schematically illustrating the persons and thedevice in the certain place;

FIG. 16 is a diagram illustrating the screen;

FIG. 17 is a diagram illustrating the screen;

FIG. 18 is a diagram schematically illustrating tomographic images of aperson's brain;

FIG. 19 is a diagram illustrating another screen;

FIG. 20 is a diagram illustrating another screen;

FIG. 21 is a diagram illustrating a specific example of graphs;

FIG. 22 is a diagram illustrating a specific example of a graph;

FIG. 23 is a diagram illustrating a setting screen;

FIG. 24 is a diagram illustrating the screen;

FIG. 25 is a diagram illustrating the screen;

FIG. 26 is a diagram illustrating the screen;

FIG. 27 is a flowchart illustrating a process in a twentieth example;

FIG. 28 is a flowchart illustrating another process in the twentiethexample;

FIG. 29 is a perspective view of the entirety of a hearable;

FIG. 30 is a diagram illustrating a biological information measuringapparatus viewed from above;

FIG. 31 is a diagram illustrating a left earphone viewed from the left;

FIG. 32 is a perspective view of a right earphone;

FIG. 33 is a perspective view of the left earphone;

FIG. 34 is a diagram illustrating the right earphone viewed from above;

FIG. 35 is an exploded perspective view of the right earphone;

FIG. 36 is a cross-sectional view of the right earphone;

FIG. 37 is a perspective view of the inside of the left earphone;

FIG. 38 illustrates brain wave data of a person in a state of liking ordisliking;

FIG. 39 illustrates brain wave data of a person in a state of surprise;

FIG. 40 illustrates brain wave data of a person who is viewing a VRimage containing a monster-like creature;

FIG. 41 illustrates brain wave data of a person performing a mental task(completing a white puzzle);

FIG. 42 illustrates brain wave data of a person receiving a headmassage;

FIG. 43 illustrates brain wave data of a person listening to music ortaking a supplement;

FIG. 44 illustrates brain wave data of a person undergoing beginning ofsleep and successive sleep states; and

FIG. 45 illustrates change in brain waves when the user experiencesitches.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

An information processing system according to an exemplary embodimentwill be described with reference to FIG. 1. FIG. 1 illustrates anexample of the configuration of the information processing systemaccording to the present exemplary embodiment.

The information processing system according to the present exemplaryembodiment includes an information processing apparatus 10, at least onebiological information measuring apparatus 12, at least one environmentinformation measuring apparatus 14, at least one device 16, and at leastone terminal apparatus 18. The number of apparatuses illustrated in FIG.1 is just an example, and the number of apparatuses used is not limitedto that illustrated in FIG. 1. The information processing system mayalso include apparatuses (e.g., an external apparatus such as a server)other than those illustrated in FIG. 1. When the device 16 is not used,the device 16 need not be included in the information processing system.When the terminal apparatus 18 is not used, the terminal apparatus 18need not be included in the information processing system.

The apparatuses included in the information processing system are eachconfigured to communicate with the other apparatuses. The communicationmay be wired communication employing cable or wireless communication.That is, the apparatuses may be physically connected to one another bycable and communicate information with one another or communicateinformation with one another through wireless communication.Short-distance wireless communication or Wi-Fi (registered trademark),for example, is used for the wireless communication. Wirelesscommunication based on another standard may be used, instead. Bluetooth(registered trademark), radio-frequency identification (RFID), ornear-field communication (NFC), for example, may be used as the shortdistance wireless communication. The apparatuses may communicate withone another through a communication path such as a local-area network(LAN) or the Internet.

The information processing apparatus 10 is, for example, a personalcomputer (PC), a tablet PC, a smartphone, a mobile phone, or anotherapparatus (e.g., a server, a robot, etc.). The information processingapparatus 10 may be a terminal apparatus carriable by a person (e.g., atablet PC, a smartphone, a mobile phone, etc.), an apparatus set on atable or the like for use, a mobile apparatus (e.g., a self-propelledapparatus), an apparatus that performs operations, or an apparatuscapable of making conversations with persons. For example, theinformation processing apparatus 10 may be a smart speaker having acommunication function and including a microphone and a speaker or arobot that makes conversations with persons and that performsoperations. The information processing apparatus 10 may be an apparatusprovided with artificial intelligence (AI). The biological informationmeasuring apparatus 12 is configured to measure biological informationregarding living things. The living things may be persons, animals otherthan persons, or plants.

For example, the biological information measuring apparatuses 12 includevarious sensors and electrodes and measures biological informationregarding a living thing. When plural biological information measuringapparatuses 12 are used, the biological information measuringapparatuses 12 may measure biological information of different types.Some or all of the biological information measuring apparatuses 12 maymeasure biological information of the same type. The biologicalinformation measuring apparatus 12 may measure biological information ofa single type or plural types.

Biological information may include various pieces of informationexhibited by a living thing. The biological information is, for example,information indicating brain activity (e.g., brain waves, cerebral bloodflow, brain magnetic field signals, etc.), information indicating pulserate, myoelectric information such as a myoelectric waveform,information regarding saliva (e.g., information indicating salivavolume), information indicating a pulse wave, information indicatingblood pressure, information indicating blood flow, informationindicating pulse, information indicating heart rate, informationindicating an electrocardiographic waveform, information indicating eyemovement, information indicating body temperature, informationindicating the amount of sweat, information indicating a line of sight,sound information, information indicating movement of a person, orinformation obtained from a body fluid (e.g., blood). Alternatively,information identified from a biomarker may be used as biologicalinformation. Alternatively, the biological information may beinformation based on a potential detected from a living thing. Forexample, the biological information may be brain waves that are a resultof measurement of minute currents caused by brain activity, anelectrocardiogram that is a result of measurement of minute currentscaused by heartbeats, an electromyogram that is a result of measurementof minute currents caused by muscle activity, or a skin potential thatis a result of measurement of minute currents caused in the skin. Theseare just examples of the biological information, and other types ofbiological information may be used, instead.

Emotion information, mental information, or psychological informationmay also be obtained as state information regarding a person byanalyzing biological information. For example, information indicating aperson's emotion, information indicating a person's mental state, orinformation indicating a person's psychological state may be obtained byanalyzing biological information regarding the person. Informationindicating a state of an animal other than a person or a plant may beobtained by analyzing biological information regarding the animal or theplant. As the state information regarding a person, there is a state inwhich the person is feeling one of three major desires, namely appetite,sleep desire, and sexual desire, and a kind of consciousnesscharacterized by feeling. More specifically, however, the stateinformation regarding a person is information regarding relief, anxiety,gratitude, astonishment, excitement, sexual arousal, curiosity, sexualcuriosity, calmness, impatience (irritability), wonder (embarrassment),happiness, luck, relaxation, tension, honor, responsibility, respect,familiarity (friendliness), longing (aspiration), desire (motivation),fear, courage, comfort, pleasure (in terms of good deeds and virtues),regret, satisfaction, dissatisfaction, remorse, disgust, shame,contempt, jealousy, guilt, intent to kill, schadenfreude, saudade,anticipation, superiority, inferiority, grudge, resentment, suffering,sadness, heartbreak, inspiration, anger, distress (agony, anguish,painfulness), resignation (giving up), despair, hope, hatred (love andhate), love, or emptiness. In addition, there are itchiness, pain, urgeto urinate, urge to defecate, hunger, satiety, and the like. Inaddition, there is information regarding a person's thought or intentionas information to be communicated between persons, such as approval,agreement, disagreement, rejection, hesitation, or confusion. Becausesuch information can be abundantly obtained from brain wave informationregarding a person, the information regarding a person's brain waves maybe conveniently used.

The brain waves are generally categorized as follows. The delta (δ) wavehas a frequency range of between 1 to 4 Hz, and corresponds to the stateof deep sleep without dreams, a non-REM sleep, decreases as the sleepbecomes shallower, and corresponds to an unconscious state. The theta(θ) wave has a frequency range of between 4 to 8 Hz, and corresponds toa resting state, dozing-off state, a light fitful sleep, an intuitivestate, a creative state, a state of accessing to memory, and insight.The low alpha (α) wave has a frequency range of between 8 to 10 Hz, andcorresponds to a relaxed state (resting and conscious but introvertedstate), appears when closing eyes, and decreases when opening eyes. Thehigh alpha (α) wave has a frequency range of between 10 to 12 Hz, andcorresponds to a relaxed state (concentrated and extroverted state),appears when closing eyes, and decreases when opening eyes. The SMR(sensorimotor rhythm) has a frequency range of between 12 to 15 Hz, andcorresponds to an in-the-zone state, a balanced state between relaxedand concentrated, appears before movement, and appears before thinking.The low beta (β) wave has a frequency range of between 15 to 20 Hz, andcorresponds to thoughts and when solving problems. The high beta (β)wave has a frequency range of between 20 to 30 Hz, and corresponds to analert state, a nervous state, a stressed state, and a worrying state.The low gamma (γ) wave has a frequency range of between 31 to 40 Hz, andcorresponds to memory formation, a higher mental activity, a highercognitive activity, a happy state. The middle gamma (γ) wave has afrequency range of between 41 to 50 Hz and corresponds to visualinformation processing. The biological information measuring apparatus12 may measure part of all of the brain waves discussed above.

The biological information measuring apparatus 12 may be an apparatus(e.g., a wearable device) that is worn by a living thing which is asubject and that measures biological information or may be an apparatusthat measures biological information without being worn by a livingthing.

A wearable device is, for example, a hearable worn on an animal's ears,a device worn on an animal's head, a device (e.g., a wristwatch-typedevice such as a smart watch) worn on an animal's wrist, arm, or finger(hereinafter referred to as a “wrist or the like”), a device worn on ananimal's neck, a devices worn on an animal's body (e.g., chest, abdomen,etc.), a device worn on a lower limb (e.g., a person's thigh, lower leg,knee, foot, ankle, etc.), a glasses-type device, or a contact lens-typedevice. The biological information measuring apparatus 12 may worn on apart other than these, instead. In addition, the biological informationmeasuring apparatus 12 may be worn on more than one part.

The hearable may be, for example, earphones, a hearing aid, anearring-type device, a clip-type device, or a device with a band or acable to be wound around the ears. The device worn on the head may be,for example, a headset with a band or a cable to be wound around thehead. The device worn on the wrist may be, for example, a device with aband or a cable to be wound around the wrist. Devices worn on otherparts may also come with bands or cables. Hearables, which usually havenatural appearances and can perform measurement, may be used especiallywhen brain wave information is measured. Hearables are suitable asdevices for measuring brain waves because the hearables have highlyversatile shapes like common earphones. Whereas hearables have naturalappearances that do not seem to be measuring brain waves at firstglance, people can easily tell that medical electroencephalographs,which usually attach electrodes to the head, are measuring brain waves.This causes hesitation in wearing medical electroencephalographs indaily life. In addition, because hearables can use means that can inputsound to a person's ears and/or receive voice information from theperson's mouth, a result of interpretation of the voice information canbe checked with sound and errors in the interpretation can be correctedwhile detecting brain wave information. Furthermore, because BMIs andbrain-computer interfaces (BCIs), which employ brain waves, are oftenused when devices are operated or persons communicate with each other,the devices need to be operated accurately and information to becommunicated needs to be correctly recognized. Because BMIs and BCIsthat employ hearables are worn on the ears, it is possible to improveaccuracy using the above-mentioned sound input means, or voiceinformation or biological information other than brain waves that hasbeen intentionally caused by a person (e.g., it is possible to checkwhether electromyographic information generated by intentionally movingthe jaw three times, eye potential information generated byintentionally moving the eyes strongly, or the like can be correctlyrecognized and use such information to, for example, determine whetherto perform an operation or communicate with another person).

The biological information measuring apparatus 12 may be an apparatus ofa contact type that measures biological information regarding a livingthing while coming into contact with the living thing or may be anapparatus of a noncontact type that measures biological informationregarding a living thing without coming into contact with the livingthing. Alternatively, the biological information measuring apparatus 12may have functions of both a contact type and a noncontact type. Thatis, the biological information measuring apparatus 12 may measurebiological information regarding a living thing while coming intocontact with the living thing and without coming into contact with theliving thing. The biological information measured through a contact withthe living thing and the biological information measured without acontact with a living thing may be of the same type or of differenttypes.

The biological information measuring apparatus 12 includes electrodesfor measuring biological information and sensors other than theelectrodes. The electrodes may come into contact with a living thing anddetect a potential, which is an example of the biological information ormay detect a potential without coming into contact with the livingthing. The sensors other than the electrodes may measure biologicalinformation while coming into contact with the living thing or withoutcoming into contact with the living thing. For example, the electrodescome into contact with an animal and detect potential indicating brainwaves of the animal. The sensors, on the other hand, may measurebiological information indicating body temperature of the animal withoutcoming into contact with the animal. This is just an example, and othertypes of biological information may be measured, instead.

For example, the biological information measuring apparatus 12 includesone or more electrodes. Plural electrodes are attached to a livingthing, for example, and detect a potential. The detected potentialincludes a biopotential, which is an example of the biologicalinformation regarding a living thing, and the biological potential isextracted from the detected potential by processing the detectedpotential. The detected potential might include noise that does notderive from the biopotential, for example, and biological potentialsfrom which the noise has been removed can be obtained by performingprocessing such as noise cancellation. The noise is, for example, apotential caused by movement of the living thing, a potential fromoutside the living thing, a potential deriving from a globalenvironment, a potential indicating biological information regardingliving things other than a measurement target, or the like. The noisemay also include potentials generated by devices such as PCs andsmartphones. Different electrodes may be used to detect a potential inaccordance with potential detection sensitivity, a measurement conditionof noise, or the like. In addition, when potentials deriving fromvarious biopotentials are collectively measured, the potentials derivingfrom the various biological potentials may be estimated and separatedfrom one another by dividing the measured potential in accordance withfrequencies corresponding to the biological potentials or dividingmeasurement time into periods where all the potentials have beencollectively measured and periods where only some of the potentials havebeen measured. In the case of the above-described hearable, for example,information indicating brain activity, information indicating pulserate, myoelectric information deriving from movement of muscles and thelike, information deriving from blood flow such as pulse wave and heartrate, and the like might be collectively measured, but these pieces ofinformation often have different frequencies or absolute values ofoutputs. These pieces of information, therefore, can be separatelyestimated through a frequency analysis of a measured signal or on thebasis of levels of output values. Brain waves, for example, can bedivided into bands of alpha, beta, theta, delta, and gamma waves. Inanother method, a hearable may perform measurement and a commerciallyavailable electroencephalograph, a commercially availableelectromyograph, and a commercially available measuring device relatingto blood flow may measure individual measurement signals plural timesunder the same condition. Pieces of biological information from whichmeasurement signal information collectively measured by the hearablederives may then be determined by analyzing the measurement signalinformation through a frequency analysis based on a Fourier transform ora wavelet transform (WT), and the measurement signal information may bedivided through statistical processing.

The biological information measuring apparatus 12 transmits the measuredbiological information to the information processing apparatus 10. Thebiological information measuring apparatus 12 may transmit biologicalinformation to the information processing apparatus 10 each time thebiological information measuring apparatus 12 has measured biologicalinformation or store biological information and transmit the biologicalinformation to the information processing apparatus 10 at predeterminedtime intervals or timings specified by a person. Alternatively, thebiological information measuring apparatus 12 may receive biologicalinformation measured by another biological information measuringapparatus 12 from the other biological information measuring apparatus12 and transmit biological information measured thereby and thebiological information measured by the other biological informationmeasuring apparatus 12 to the information processing apparatus 10.

The biological information measuring apparatus 12 may analyze biologicalinformation measured thereby or another biological information measuringapparatus 12 and transmit information indicating a result of theanalysis to the information processing apparatus 10. For example, thebiological information measuring apparatus 12 may include a processor,and the processor may analyze biological information. The informationprocessing apparatus 10 or another apparatus may conduct the analysis,instead.

The biological information measuring apparatus 12 may include a batteryand be driven by power supplied from the battery or may receive powerfrom another apparatus (e.g., the information processing apparatus 10)and be driven.

For example, the information processing apparatus 10 receives biologicalinformation from the biological information measuring apparatus 12,analyzes, stores, and outputs the biological information, and stores andoutputs information indicating a result of the analysis of thebiological information. The biological information measuring apparatus12 or another apparatus may analyze the biological information, instead.The outputting of biological information is, for example, displaying ofthe biological information or outputting of the biological informationas sound information. The outputting of information indicating a resultof an analysis of biological information is, for example, displaying ofthe information indicating the result of the analysis or outputting ofthe result of the analysis as sound information. The informationprocessing apparatus 10 may transmit the biological information and theinformation indicating the result of the analysis to another apparatus,instead.

The information processing apparatus 10 may include the at least onebiological information measuring apparatus 12. That is, the at least onebiological information measuring apparatus 12 may be incorporated intothe information processing apparatus 10 to form a single apparatus. Theinformation processing apparatus 10 including the at least onebiological information measuring apparatus 12 may be worn by a livingthing and measure biological information. That is, the informationprocessing apparatus 10 may be a wearable device. For example, theinformation processing apparatus 10 may be an apparatus worn on aperson's head (e.g., forehead etc.), a hearable worn on a person's ears(e.g., earphones, headphones, etc.), an apparatus worn on a person'sarm, hand, wrist, or finger (e.g., a wristwatch-type device such as asmart watch), an apparatus worn on a person's neck, an apparatus worn ona person's body (e.g., abdomen, chest, etc.), or an apparatus worn onone of a person's lower limbs (e.g., thigh, lower leg, knee, foot,ankle, etc.). The information processing apparatus 10 may be a healthappliance or the like worn on a person's arm, hand, body, or lower limb.The information processing apparatus 10 may be worn on a part other thanthese parts, instead.

The information processing apparatus 10 and the biological informationmeasuring apparatus 12 may be separate apparatuses. For example, theinformation processing apparatus 10 may be an apparatus such as a robot,a smart speaker, or a server, and the biological information measuringapparatus 12 may be a wearable device worn by a living thing.

The environment information measuring apparatus 14 is configured tomeasure environment information regarding a surrounding environment of aliving thing and the environment information measuring apparatus 14. Theenvironment information measuring apparatus 14 is, for example, a camerathat is an image capture device, a microphone that collects sounds, atemperature sensor that measures temperature, a humidity sensor thatmeasures humidity, an odor sensor that measures odor, an illuminancesensor that measures brightness, an infrared sensor, a pressure sensorsuch as an air pressure sensor, a position sensor that detects movementof a target, a human detection sensor, or the like. At least one ofthese sensors may be included in the information processing system asthe environment information measuring apparatus 14.

For example, a camera captures images of a surrounding environment ofthe information processing apparatus 10 and other places to generateimage data indicating the surrounding environment and image dataindicating the other places. The image data may be moving image data orstill image data. The image data obtained by the camera is an example ofenvironment information regarding an environment included in an imagecapture range of the camera. Image data indicating a living thinggenerated by capturing an image of the living thing using the camera maybe used as biological information regarding the living thing. Forexample, movement of the living thing (e.g., a person), a figure of theliving thing, or the like detected from the image data is an example ofbiological information regarding the living thing. In this sense, thecamera is an example of the biological information measuring apparatus12 that measures biological information regarding a living thing.

Sounds around a microphone (e.g., a person's voice and other sounds) areinput to the microphone, and the microphone generates sound data. Thesound data indicating the sounds input to the microphone is an exampleof environment information regarding a surrounding environment of themicrophone. The sound data indicating a person's voice input to themicrophone may be used as biological information regarding the person.In this sense, the microphone is an example of the biologicalinformation measuring apparatus 12 that measures biological informationregarding a person. Sound data indicating cries of an animal other thana person may be used as biological information regarding the animal.

Data measured by a temperature sensor, a humidity sensor, an odorsensor, an illuminance sensor, an infrared sensor, a pressure sensor, aposition sensor, a human detection sensor, or the like is an example ofthe environment information. Data measured by these sensors from aliving thing may be used as biological information regarding the livingthing. In this sensor, these sensors are examples of the biologicalinformation measuring apparatus 12 that measures biological informationregarding a living thing.

The environment information may be information indicating the size of aroom in which a living thing exists, information indicating the size ofa room in which the device 16 is provided, information indicating thenumber of windows provided for a room, or the like. The environmentinformation is not limited to information measured by the environmentinformation measuring apparatus 14 and may be predetermined informationor premeasured information, instead.

The at least one environment information measuring apparatus 14 may beincluded in the information processing apparatus 10. The device 16 is anexample of an operation target. The device 16 is, for example, a PC, atablet PC, a smartphone, a mobile phone, a robot (e.g., a humanoidrobot, an animal robot, a cleaning robot, etc.), a projector, a displaydevice such as a liquid crystal display, a recording device, a playbackdevice, an image capture device such as a camera, a refrigerator, a ricecooker, a microwave oven, a coffee maker, a vacuum cleaner, a washingmachine, an air conditioner, a lighting device, a clock, a surveillancecamera, an automobile, a motorcycle, an airplane (e.g., an unmannedaerial vehicle such as a drone), a game machine, a gas range, anelectronic toilet seat, a ventilation fan, a call bell, an entrancemonitor, an elevator, a door, a window, or one of various sensingdevices (e.g., a temperature sensor, a humidity sensor, a voltagesensor, a current sensor, etc.). A concept of the device 16 may includedevices in general. For example, the concept of the device 16 accordingto the present exemplary embodiment may include information devices,video devices, audio devices, and other devices.

The device 16 includes a communication unit, which is a communicationinterface, a memory storing information, and a processor that controlsthe operation thereof. The device 16 may include user interfaces (UIs)such as a display and an operation unit. The device 16 may transmitdevice identification information for identifying the device 16 to theinformation processing apparatus 10. The device identificationinformation is, for example, an identifier (ID), a name, a model number,or an address (e.g., a media access control (MAC) address, an Internetprotocol (IP) address, etc.) of the device 16.

In the present exemplary embodiment, an operation target is operated onthe basis of biological information regarding at least one living thing(e.g., at least one person). For example, the information processingapparatus 10 obtains biological information regarding a living thingfrom the biological information measuring apparatus 12 and operates anoperation target on the basis of the obtained biological information.For example, the information processing apparatus 10 obtains biologicalinformation regarding a living thing from the biological informationmeasuring apparatus 12, identifies an operation to be performed on theoperation target on the basis of the obtained biological information,and operates the operation target in accordance with the identifiedoperation. A process for identifying an operation may be performed bythe biological information measuring apparatus 12 or another apparatus(e.g., a server, the device 16, the terminal apparatus 18, etc.),instead. An apparatus (e.g., a server, the terminal apparatus 18, etc.)other than the information processing apparatus 10 may operate theoperation target, instead. For example, the information processingapparatus 10 may identify an operation on the basis of biologicalinformation, and an apparatus other than the information processingapparatus 10 may operate the operation target in accordance with theoperation identified by the information processing apparatus 10.

The operation target may be hardware or software, or a combination ofhardware and software.

Hardware to be operated is not particularly limited, and may be, forexample, the device 16 or an apparatus other than the device 16 (e.g.,the information processing apparatus 10, the biological informationmeasuring apparatus 12, the environment information measuring apparatus14, a server, etc.). Plural pieces of hardware may be determined asoperation targets, instead.

Software to be operated is not particularly limited, and may be, forexample, an application program, an operating system (OS), or the like.The software to be operated may be software installed on the informationprocessing apparatus 10, the device 16, the terminal apparatus 18, oranother apparatus (e.g., the biological information measuring apparatus12, the environment information measuring apparatus 14, a server, etc.)or software provided through a communication path such as the Internet.Plural pieces of software may be determined as operation targets,instead. When plural pieces of software are to be operated, the piecesof software may be installed on the same piece of hardware or differentpieces of hardware.

The configuration of the information processing apparatus 10 will bedescribed in detail hereinafter with reference to FIG. 2. FIG. 2illustrates an example of the hardware configuration of the informationprocessing apparatus 10.

The information processing apparatus 10 includes, for example, acommunication device 20, a UI 22, a memory 24, and a processor 26. Theinformation processing apparatus 10 may also include other components.

The communication device 20 is a communication interface including acommunication chip or a communication circuit and has a function oftransmitting information to other apparatuses and a function ofreceiving information transmitted from other apparatuses. Thecommunication device 20 may have a wireless communication function or awired communication function. The communication device 20 maycommunicate with the other apparatuses through, for example,short-distance wireless communication or a communication path such as aLAN or the Internet.

The UI 22 includes at least a display or an operation device. Thedisplay is liquid crystal display, an electroluminescent (EL) display,or the like. The operation device is a keyboard, input keys, anoperation panel, or the like. The UI 22 may be a touch panel or the likeincluding both a display and an operation device. The UI 22 may alsoinclude a microphone and a speaker. The information processing apparatus10 need not include the UI 22.

The memory 24 is a device for achieving one or more storage areasstoring various pieces of information. The memory 24 is, for example, ahard disk drive, one of various memories (e.g., a random-access memory(RAM), a dynamic random-access memory (DRAM), a read-only memory (ROM),etc.), another storage device (e.g., an optical disc etc.), or anyselective combination of these. At least one memory 24 is included inthe information processing apparatus 10.

The memory 24 stores management information. The management informationis information for identifying, on the basis of biological information,an operation to be performed on an operation target.

The processor 26 is configured to control the operation of the othercomponents of the information processing apparatus 10. The processor 26may include a memory.

For example, predetermined reference biological information andoperation information indicating operations to be performed on operationtargets are registered in the management information in advance whilebeing associated with each other. The reference biological informationmay be biological information estimated to be caused in an animal (e.g.,a person or another animal) that is performing an operation associatedwith the reference biological information, biological informationestimated to be caused in an animal that is requesting or desiresexecution of the operation, or biological information estimated to becaused in a plant. In other words, the reference biological informationis biological information indicating an operation to be performed on anoperation target. The operation information indicating an operation mayinclude information indicating an operation target to be operated inaccordance with the operation.

The reference biological information and the operation information maybe registered in the management information for each living thing whilebeing associated with each other. For example, the reference biologicalinformation and the operation information may be registered in themanagement information for each type of living thing while beingassociated with each other. For example, management information forpersons, management information for animals other than persons, andmanagement information for plants may be created and stored in thememory 24. In the management information for persons, referencebiological information estimated to be measured in the persons andoperation information are registered while being associated with eachother. Reference biological information and operation information may beregistered in the management information for each person (e.g., for eachuser) while being associated with each other. In the managementinformation for animals other than persons, reference biologicalinformation estimated to be measured in the animals and operationinformation are registered while being associated with each other.Management information may be created for each type of animal other thanpersons. In the management information for plants, reference biologicalinformation estimated to be measured in plants and operation informationare registered while being associated with each other. Managementinformation may be created for each type of plant.

The operation information may be information indicating an operation tobe performed on a power supply of the device 16, information indicatingan operation to be performed for a function level of the device 16, orinformation indicating both an operation to be performed on the powersupply of the device 16 and an operation to be performed for thefunction level.

The operation to be performed on the power supply of the device 16 is anoperation for turning on or off the device 16. The operation informationregarding the operation to be performed on the power supply isinformation indicating an operation for turning on or off the device 16.Biological information associated with the operation informationregarding the operation to be performed on the power supply isbiological information corresponding to turning on or off of the device16. Operation information regarding an operation to be performed on thepower supply of the device 16 and reference biological information maybe registered in management information for each type of living thing orfor each living thing while being associated with each other.

The operation to be performed for the function level of the device 16 isan operation for setting the function level of the device 16. Theoperation information regarding the operation to be performed for thefunction level is information indicating an operation for setting thefunction level of the device 16. Biological information associated withthe operation information regarding the operation to be performed forthe function level is biological information corresponding to thefunction level of the device 16. Operation information regarding anoperation to be performed for the function level of the device 16 andreference biological information may be registered in managementinformation for each type of living thing or for each living thing whilebeing associated with each other.

The function level is, for example, a level relating to performance oroutput of the device 16. Specific examples of the function level includesetting temperature, wind volume, wind direction, and presence orabsence of a dehumidification function of an air conditioner, luminanceof a display device or a lighting device, sound volume of a speaker,traveling speed of a self-propelled apparatus (e.g., a robot, aself-propelled vacuum cleaner, etc.), setting values of a device such asan image capture device, a recording device, a playback device, settingvalues of an apparatus such as a refrigerator, a rice cooker, amicrowave oven, and setting values of one of various sensing devices.These are just examples, and another setting value or the like may beused as the function level, instead.

The management information may be stored in an apparatus other than theinformation processing apparatus 10 instead of, or as well as, theinformation processing apparatus 10.

The processor 26 is configured to obtain biological informationregarding a living thing and operate an operation target on the basis ofthe biological information.

When the biological information measuring apparatus 12 measuresbiological information in a living thing, for example, the biologicalinformation measuring apparatus 12 transmits the biological informationto the information processing apparatus 10. The processor 26 receivesthe biological information and identifies an operation to be performedon an operation target on the basis of the biological information. Theprocessor 26 operates the operation target in accordance with theidentified operation. The processor 26 may identify the operation targetand the operation to be performed on the operation target on the basisof the biological information regarding the living thing, instead. Inanother example, the operation target may be determined in advance. Forexample, a person may specify the operation target. In this case, theprocessor 26 identifies an operation to be performed on the specifiedoperation target on the basis of the biological information regardingthe living thing.

If the operation target is the device 16, the processor 26 transmitscontrol information including operation information indicating anidentified operation to the device 16 to operate the device 16. Thedevice 16 to be operated operates in accordance with the controlinformation. The same holds when hardware other than the device 16 isthe operation target.

If the operation target is software, the processor 26 operates thesoftware in accordance with the identified operation. For example, theprocessor 26 activates the software or performs processing using thesoftware. If the software is installed on the information processingapparatus 10, the processor 26 operates the software installed on theinformation processing apparatus 10. If the software is installed on anapparatus other than the information processing apparatus 10, theprocessor 26 transmits control information including operationinformation indicating the operation to the apparatus on which thesoftware is installed to operate the software.

For example, the processor 26 compares the biological informationmeasured in the living thing and the reference biological informationregistered in the management information and searches for a piece ofreference biological information whose difference from the biologicalinformation is within an allowable range. The processor 26 identifies anoperation to be performed on an operation target associated with thefound piece of reference biological information. The operation targetand the operation to be performed on the operation target are thusidentified. The allowable range is determined in advance. The user maychange the allowable range. The allowable range may be defined for eachtype of living thing or for each living thing.

If plural pieces of reference biological information whose differencesfrom the biological information measured in the living thing are withinthe allowable range are found, the processor 26 may identify, among theplural pieces of reference biological information, a piece of referencebiological information whose difference from the biological informationis the smallest and then identify an operation associated with theidentified piece of reference biological information or may identifyoperations associated with the plural pieces of reference biologicalinformation.

The reference biological information may be information indicating acharacteristic component of biological information. In this case, theprocessor 26 may extract a characteristic component from the biologicalinformation measured in the living thing and search for a piece ofreference biological information including a component whose differencefrom the extracted component is within an allowable range. When brainwaves are used as biological information, for example, the processor 26may extract a characteristic component from brain waves of an animal andestimate an operation indicated by the brain waves by analyzing thecharacteristic component.

The reference biological information may be used as a threshold,instead. When the reference biological information is a waveform, forexample, the amplitude of a peak in the waveform is used as a threshold.In this case, the processor 26 searches for a piece of referencebiological information that has a waveform approximate or similar to awaveform indicated by the biological information measured in the livingthing and with which the measured biological information exceeds athreshold and identifies an operation associated with the found piece ofreference biological information. That is, if the waveform indicated bythe measured biological information is approximate or similar to awaveform indicated by a certain piece of reference biologicalinformation and the amplitude of a peak in a waveform indicated by themeasured biological information is larger than or equal to a threshold,which is the amplitude of a peak in the waveform indicated by the pieceof reference biological information, the processor 26 identifies anoperation associated with the piece of reference biological information.

When an operation target is operated using information indicating brainactivity of an animal, the biological information measuring apparatus 12that measures the brain activity and the information processingapparatus 10 may together construct a BMI. In general, “BMI” is ageneral term for devices and the like that serve as interfaces betweenthe brain and a computer or the like through detection of brain waves,stimulation of the brain, or the like. When the brain is connected to acomputer, such devices are called “BCIs”. In the present exemplaryembodiment, usage examples where devices are operated or personscommunicate with each other (communication of information regarding aperson's thought or intention or state information etc.) usingbiological information including brain waves will be described. Theseusage examples will be regarded as part of BMIs hereinafter. An invasivemethod or a non-invasive method may be employed for a BMI. In this case,the processor 26 operates an operation target on the basis of brainactivity (e.g., brain waves etc.). In order to operate an operationtarget, the processor 26 may extract a characteristic component frombrain waves and operate the operation target on the basis of theextracted characteristic component. In order to extract a characteristiccomponent from brain waves, a fast Fourier transform (FFT), a WT,time-frequency distribution (TFD), an eigenvector method (EM), anautoregressive model (ARM), or the like may be used. As a method forconnecting brain waves and an operation to be performed on an operationtarget with each other using a feature vector obtained by extracting afeature, or as a method for connecting a communication target andinformation to be communicated with each other in communication betweenpersons, for example, an independent component analysis (ICA), k-meansclustering, a support vector machine (SVM), a convolutional neuralnetwork, or the like may be used.

Alternatively, the processor 26 may receive identification informationregarding an operation target transmitted from the operation target andidentify the operation target. For example, the processor 26 transmits arequest to obtain identification information to an operation target andthen obtains the identification information transmitted from theoperation target in response to the request. In another case, hardwareconnected to the information processing apparatus 10 and capable ofcommunicating with the information processing apparatus 10, for example,may transmit identification information regarding the hardware oridentification information regarding software installed thereon to theinformation processing apparatus 10. The processor 26 receives theidentification information transmitted in this manner.

The processing performed by the processor 26 may be performed by anapparatus other than the information processing apparatus 10 (e.g., thebiological information measuring apparatus 12, the environmentinformation measuring apparatus 14, the device 16, the terminalapparatus 18, or the like), instead. In addition, the informationprocessing apparatus 10 may perform a part of the processing, and anapparatus other than the information processing apparatus 10 may performanother part of the processing. For example, an apparatus other than theinformation processing apparatus 10 may conduct an analysis ofbiological information (e.g., the process for identifying an operation),and the information processing apparatus 10 may operate the operationtarget.

The configuration of the terminal apparatus 18 will be described indetail hereinafter with reference to FIG. 3. FIG. 3 illustrates anexample of the hardware configuration of the terminal apparatuses 18.

The terminal apparatuses 18 includes, for example, a communicationdevice 28, a UI 30, a memory 32, and a processor 34.

The communication device 28 is a communication interface including acommunication chip or a communication circuit and has a function oftransmitting information to other apparatuses and a function ofreceiving information transmitted from other apparatuses. Thecommunication device 28 may have a wireless communication function or awired communication function. The communication device 28 maycommunicate with the other apparatuses through, for example,short-distance wireless communication or a communication path such as aLAN or the Internet.

The UI 30 includes at least a display or an operation device. Thedisplay is liquid crystal display, an EL display, or the like. Theoperation device is a keyboard, input keys, an operation panel, or thelike. The UI 30 may be a touch panel or the like including both adisplay and an operation device. The UI 30 may also include a microphoneand a speaker.

The memory 32 is a device for achieving one or more storage areasstoring various pieces of information. The memory 32 is, for example, ahard disk drive, one of various memories (e.g., a RAM, a DRAM, a ROM,etc.), another storage device (e.g., an optical disc etc.), or anyselective combination of these. At least one memory 32 is included inthe terminal apparatus 18.

The processor 34 is configured to control the other components of theterminal apparatus 18. The processor 34 may include a memory.

Specific examples of the management information will be describedhereinafter.

FIG. 4 illustrates an example of a management table, which is an exampleof the management information. Data regarding the management table isstored in the memory 24. The data regarding the management table neednot be stored in the memory 24 and may be stored in an apparatus otherthan the information processing apparatus 10 (e.g., a server), instead.

In the management table, IDs, reference brain waves, and operationinformation indicating operations to be performed on an operation targetare associated with one another in advance.

The IDs are information for managing information registered in themanagement table.

The reference brain waves are an example of the reference biologicalinformation. The reference brain waves are brain waves estimated to bemeasured in a person. Although brain waves are used as an example of thereference biological information, biological information other thanbrain waves may be used as the reference biological information,instead.

The reference brain waves are defined, for example, through statisticalprocessing. The reference brain waves are brain waves estimated to becaused in a person who is performing an operation associated with thereference brain waves or brain waves estimated to be caused in a personwho is requesting execution of the operation.

The reference brain waves may be brain waves in a certain frequency bandor include brain waves from different frequency bands.

In the example illustrated in FIG. 4, the operation target is the device16, which is an example of the hardware. The operation target may behardware other than the device 16 or software, instead.

The operation information includes device identification information foridentifying the device 16 to be operated and information indicating anoperation to be performed on the device 16. The operation may be, forexample, an operation for turning on or off the device 16, an operationfor setting a function level of the device 16, or a combination of anoperation for turning on or off the device 16 and an operation forsetting a function level.

For example, reference brain waves whose ID is “1” are brain wavesindicating an operation for starting a cooling function of an airconditioner. Reference brain waves whose ID is “2” are brain wavesindicating an operation for stopping the cooling function of the airconditioner.

If brain waves whose difference from the reference brain waves whose IDis “1” is within an allowable range are measured in a person, forexample, the processor 26 identifies the operation for starting thecooling function of the air conditioner associated with the referencebrain waves. The processor 26 then transmits control informationincluding information indicating the operation for starting the coolingfunction of the air conditioner to the air conditioner. The airconditioner operates in accordance with the control information. As aresult, the cooling function of the air conditioner starts.

Alternatively, the processor 26 may calculate a degree of similaritybetween brain waves measured in a person and reference brain waves anddetermine whether the degree of similarity is higher than or equal to athreshold. The threshold is a value corresponding to the allowablerange. If the degree of similarity between brain waves measured in aperson and reference brain waves is higher than or equal to thethreshold, the processor 26 determines that the brain waves measured inthe person and the reference brain waves are similar to each other. Thatis, the processor 26 determines that a difference between the brainwaves measured in the person and the reference brain waves is within theallowable range. If brain waves whose degree of similarity to thereference brain waves whose ID is “1” are measured in a person, theprocessor 26 identifies the operation for starting the cooling functionof the air conditioner.

Although the reference brain waves, which are an example of biologicalinformation, are registered in the management table in the exampleillustrated in FIG. 4, emotion information, mental information,psychological information, or the like obtained by analyzing biologicalinformation may be registered in the management table instead of thereference brain waves. For example, information indicating a sensationsuch as hotness or coldness, a mental state, or a psychological statemay be registered in the management table as the reference informationinstead of the reference brain waves. The processor 26 identifies aperson's emotion, mental state, or psychological state by analyzingbrain waves measured in the person, identifies reference informationindicating the identified emotion, mental state, or psychological state,and then identifies, in the management table, an operation associatedwith the identified reference information.

For example, the difference in the brain waves between the state ofliking and disliking is detected based on the difference in thecorresponding brain wave data, as illustrated in FIG. 38. In FIG. 38,when a person is writing what they like, the brain waves stay within aspecific region (e.g. “like-zone”), while when a person is writing whatthey do not like, some of the brain waves enter a different region (e.g.“dis-like zone”).

As another example, as illustrated in FIG. 39, a mental state of asurprise can be detected. For instance, a user may encounter aclose-call situation during driving. In such a situation, the brake maybe automatically applied and evasive action may be automatically takenbased on the driving situation detected by sensors or cameras and thedetected brain-state of the user.

A specific example will be described. Reference information indicating asensation such as hotness, a mental state, or a psychological state isregistered in the management table while being associated with operationinformation indicating the operation for starting the cooling functionof the air conditioner. The processor 26 analyzes biological information(e.g., brain waves) measured in a person and identifies the person'semotion, mental state, or psychological state. If the person's emotionindicates a sensation such as hotness, for example, the processor 26identifies the operation for starting the cooling function of the airconditioner, which is associated with reference information indicatinghotness.

The processor 26 may thus identify an operation on the basis ofbiological information or on the basis of one of various pieces ofinformation (e.g., emotion information, mental information,psychological information, etc.) obtained from the biologicalinformation. The same holds in the following examples.

Reference biological information and operation information may beregistered in the management table for each living thing while beingassociated with each other. Biological information measured inindividual persons, for example, may be registered in the managementtable as reference biological information regarding the persons. As foranimals and living things other than persons, biological informationmeasured in individual animals and living things may be registered inthe management table as reference biological information regarding theanimals and the living things.

FIG. 5 illustrates an example of a management table in which specificreference biological information regarding individual persons isregistered. In the management table illustrated in

FIG. 5, IDs, reference brain waves, which are an example of thereference biological information, operation information, and userinformation are associated with one another. The user information isinformation for identifying users, who are the persons (e.g., usernames,user IDs, etc.)

Reference brain waves associated with user information are brain wavesmeasured in a user indicated by the user information when the user wasperforming an operation associated with the reference brain waves orbrain waves measured in the user when the user was requesting theoperation. Reference brain waves measured in each user are registered inthe management table in advance.

For example, when user A manually starts the cooling function of the airconditioner, the biological information measuring apparatus 12 measuresbrain waves of user A. The measured brain waves are registered to themanagement table as reference brain waves indicating an operationperformed by user A to start the cooling function of the airconditioner. In this case, the measured reference brain waves of user A,operation information indicating the operation for starting the coolingfunction of the air conditioner, and user information for identifyinguser A are registered in the management table while being associatedwith one another. The information processing apparatus 10 may performthe registration, or another apparatus may perform the registration. Inthe example illustrated in FIG. 5, these pieces of information areregistered as information whose ID is “1”. The same holds for otheroperations and users.

Alternatively, such registration may be performed plural times, and anaverage of measured brain waves may be registered as reference brainwaves. For example, a process in which user A manually starts thecooling function of the air conditioner and the biological informationmeasuring apparatus 12 measures brain waves caused in user A may beperformed plural times, and an average of the measured brain waves maybe registered to the management table as reference brain waves of userA.

If brain waves whose difference from the reference brain waves whose IDis “1” is within an allowable range are measured in user A with user Ahaving logged in to the information processing apparatus 10, forexample, the processor 26 transmits control information including theoperation information whose ID is “1” to the air conditioner to startthe cooling function of the air conditioner. More specifically, if thebiological information measuring apparatus 12 measures brain waves afteruser A logs in to the information processing apparatus 10, the processor26 searches for reference brain waves that are registered in themanagement table and that are associated with the user information foridentifying the login user A. Because reference brain waves whose ID is“1” and reference brain waves whose ID is “3” are registered in themanagement table as reference brain waves of user A in the exampleillustrated in FIG. 5, the processor 26 finds these reference brainwaves. If a difference between the measured brain waves and thereference brain waves whose ID is “1” is within an allowable range, theprocessor 26 transmits control information including the operationinformation whose ID is “1” to the air conditioner to start the coolingfunction of the air conditioner. If a difference between the measuredbrain waves and the reference brain waves whose ID is “3” is within anallowable range, the processor 26 transmits control informationincluding operation information whose ID is “3” to the air conditionerto start the cooling function of the air conditioner.

In another example, if brain waves whose difference from the referencebrain waves whose ID is “1” is within the allowable range are measuredin user A with information indicating that a user who is operating thedevice 16 is user A set to the information processing apparatus 10, theprocessor 26 may transmit control information including the operationinformation whose ID is “1” to the air conditioner to start the coolingfunction of the air conditioner. More specifically, if the biologicalinformation measuring apparatus 12 measures brain waves with theinformation indicating that the user who is operating the device 16 isuser A set to the information processing apparatus 10, the processor 26searches for reference brain waves that are registered in the managementtable and that are associated with the user information for identifyinguser A, who is the user who is operating the device 16. If a differencebetween the measured brain waves and the reference brain waves whose IDis “1” is within the allowable range, the processor 26 transmits controlinformation including the operation information whose ID is “1” to theair conditioner to start the cooling function of the air conditioner.The user who is operating the device 16 may be set to the informationprocessing apparatus 10, for example, by a user.

As for users other than user A, information is registered to themanagement table in the same manner as for user A. For example,information associated with an ID “2” is information regarding anoperation at a time when user B has started the cooling function of theair conditioner. Information associated with the ID “3” is informationregarding an operation at a time when user A has stopped the coolingfunction of the air conditioner.

Although operation information indicating operations for starting andstopping the function of the device 16 is registered in the managementtables illustrated in FIGS. 4 and 5, operation information indicating afunction level of the device 16 may be registered in the managementtable, instead.

FIG. 6 illustrates another example of the management table. In amanagement table illustrated in FIG. 6, reference biological informationand operation information are registered for each user while beingassociated with each other. In the example illustrated in FIG. 6, brainwaves and body temperature are used as the reference biologicalinformation. In the management table, therefore, IDs, reference brainwaves, which are an example of the reference biological information,reference body temperature, which is another example of the referencebiological information, operation information, and user information areassociated with one another.

If brain waves whose difference from reference brain waves whose ID is“1” is within an allowable range are measured in user A and a bodytemperature higher than or equal to a threshold is measured in user A,for example, the processor 26 identifies an operation indicated byoperation information whose ID is “1” as an operation to be performed onthe device 16. The processor 26 then transmits control informationincluding the operation information whose ID is “1” to the airconditioner to start the cooling function of the air conditioner.

Although the reference body temperature, the reference brain waves, andthe operation information are associated with one another for eachperson in the example illustrated in FIG. 6, the reference bodytemperature, the reference brain waves, and the operation informationneed not be associated with one another for each person.

FIG. 7 illustrates another example of the management table. In amanagement table illustrated in FIG. 7, reference brain waves, referenceenvironment information, and operation information are registered foreach user while being associated with one another. Although referencebrain waves are used as an example of the reference biologicalinformation, another piece of biological information may be uses as thereference biological information, instead.

The reference environment information indicates an environment in whichthe reference brain waves associated therewith are estimated to bemeasured. Although room temperature is used as the reference environmentinformation, another piece of environment information may be used as thereference environment information, instead.

If brain waves whose difference from reference brain waves whose ID is“1” is within an allowable range are measured in user A and a roomtemperature at a time when the brain waves have been measured is higherthan or equal to a threshold, for example, the processor 26 identifiesan operation indicated by operation information whose ID is “1” as anoperation to be performed on the device 16. The operation to beperformed on the device 16 is thus identified on the basis of brainwaves measured in a user and a room temperature at a time when the brainwaves have been measured.

Although the reference biological information, the reference environmentinformation, and the operation information are associated with oneanother for each person in the example illustrated in FIG. 7, thereference biological information, the reference environment information,and the operation information need not be associated with one anotherfor each person. Examples will be described hereinafter.

First Example

A first example will be described hereinafter. In the first example,biological information measured in plural living things in an actualplace is used. An example will be described hereinafter in which thedevice 16 is operated on the basis of biological information measured inplural persons gathering in a room.

FIG. 8 schematically illustrates a room 36. There are users A, B, and C,who are the persons, in the room 36. The device 16 is provided in theroom 36. The number of users and the number of devices 16 are examples,and plural devices 16 may be provided in the room 36, instead.

Plural biological information measuring apparatuses 12 measurebiological information regarding the users. For example, users A, B, andC wear biological information measuring apparatuses that are wearabledevices, and the biological information measuring apparatuses 12 measurebiological information regarding users A, B, and C. Biologicalinformation measuring apparatuses 12 that are not worn by the users maymeasure biological information regarding the users in addition to, orinstead of, the biological information measuring apparatuses 12 worn bythe users. One or more types of biological information may be measuredin the users. It is assumed here, for example, that brain waves aremeasured in users A, B, and C. Biological information other than brainwaves may be measured, instead.

For example, the biological information measuring apparatuses worn bythe users in the room 36 transmit the biological information (e.g.,brain wave signals) regarding the users to the information processingapparatus 10. In the example illustrated in FIG. 8, the biologicalinformation measuring apparatuses 12 worn by users A, B, and C transmitbiological information (e.g., brain wave signals) regarding users A, B,and C, respectively, to the information processing apparatus 10.

The processor 26 of the information processing apparatus 10 operate thedevice 16 on the basis of brain waves of users A, B, and C. For example,the processor 26 calculates an average of the brain waves measured inusers A, B, and C and identifies an operation to be performed on thedevice 16 on the basis of the average. For example, the processor 26refers to the management table illustrated in FIG. 4, identifiesreference brain waves whose difference from the average is within anallowable range, and then identifies an operation associated with theidentified reference brain waves. The processor 26 then operates thedevice 16 in accordance with the identified operation. For example, theprocessor 26 calculates an average of amplitudes of the brain waves,identifies reference brain waves whose difference from brain waves whoseamplitude is the average is within an allowable range, and thenidentifies an operation.

In another example, if the operation to be performed on the device 16 issetting of a function level, the processor 26 may separately identifyfunction levels of the device 16 on the basis of the brain waves ofusers A, B, and C. The processor 26 then calculates an average of thefunction levels and operates the device 16 in accordance with theaverage. If the device 16 is an air conditioner having a coolingfunction, and if an operation identified on the basis of the brain wavesof user A is to change setting temperature of the cooling function ofthe air conditioner to 26C, an operation identified on the basis of thebrain waves of user B is to change the setting temperature of thecooling function of the air conditioner to 27C, and an operationidentified on the basis of the brain waves of user C is to change thesetting temperature of the cooling function of the air conditioner to28C, for example, the processor 26 calculates an average of thetemperatures to be set as the setting temperature. The processor 26 thensets the setting temperature of the cooling function of the airconditioner to the average.

If the operation to be performed on the device 16 is turning on or off,the processor 26 may separately identify turning on or off on the basisof the brain waves of users A, B, and C. The processor 26 then determinewhether to turn on or off the device 16 by a majority vote between usersA, B, and C and turns on or off the device 16 in accordance with aresult of the determination.

Operation Based on Distances Between Device 16 and Users

The processor 26 of the information processing apparatus 10 may identifythe operation to be performed on the device 16 on the basis ofbiological information (e.g., a brain wave signal) regarding a userwhose distance from the device 16 to be operated is smaller than orequal to a threshold and then operate the device 16 in accordance withthe operation. Positions of the device 16 and the users are identifiedusing a global positioning system (GPS) or another communicationtechnique, for example, and positional information regarding the device16 and the users is transmitted to the information processing apparatus10. If the users carry biological information measuring apparatuses 12or terminal apparatuses 18, for example, positions of the biologicalinformation measuring apparatuses 12 or the terminal apparatuses areidentified using a GPS or the like, and positional informationindicating the positions is transmitted to the information processingapparatus 10 as positional information indicating the positions of theusers.

If distances between users A and B and the device 16 are smaller than orequal to the threshold and a distance between user C and the device 16exceeds the threshold, for example, the processor 26 of the informationprocessing apparatus 10 calculates an average of brain waves of users Aand B, identifies the operation to be performed on the device 16 on thebasis of the average, and operates the device 16 in accordance with theidentified operation. Even if a brain wave signal of user C istransmitted to the information processing apparatus 10, the processor 26identifies the operation to be performed on the device 16 without usingbrain waves of user C. In doing so, the device 16 is operated on thebasis of biological information regarding users whose distances from thedevice 16 are smaller than or equal to the threshold.

Operation Based on Distances Between Information Processing Apparatus 10and Users

The processor 26 of the information processing apparatus 10 may identifythe operation to be performed on the device 16 on the basis ofbiological information regarding a user whose distance from theinformation processing apparatus 10 is smaller than or equal to athreshold and operate the device 16 in accordance with the operation,instead. If the information processing apparatus 10 is a smart speakeror the like and provided in the room 36, for example, the processor 26may operate the device 16 on the basis of biological informationregarding a user whose distance from the information processingapparatus 10 is smaller than or equal to the threshold.

Weighting

The processor 26 of the information processing apparatus 10 may weighthe brain waves of the users, calculate an average of the weighted brainwaves, and identify the operation to be performed on the device 16 onthe basis of the average.

For example, a weighting coefficient is set for each user, andinformation indicating the weighting coefficient for each user is storedin the memory 24 of the information processing apparatus 10. Forexample, each user may determine the weighting coefficient forhimself/herself, or a selected one of users A, B, and C may determinethe weighting coefficients for all the users. The information indicatingthe weighting coefficient for each user may be, for example, transmittedfrom the terminal apparatuses 18 owned by the users to the informationprocessing apparatus 10 or input to the information processing apparatus10 using the UI 22 of the information processing apparatus 10.

The weighting coefficient for each user may be determined on the basisof an attribute of the user. The attribute is, for example, age, gender,height, weight, or the like. A weighting coefficient is determined inadvance for each attribute, and information indicating the weightingcoefficient for each attribute is stored in the memory 24 of theinformation processing apparatus 10 in advance. For example, informationindicating attributes of the users is transmitted to the informationprocessing apparatus 10 from the terminal apparatuses 18 owned by theusers, and the processor 26 of the information processing apparatus 10determines the weighting coefficients for the users on the basis of theinformation indicating the attributes of the users.

Alternatively, the weighting coefficients may be determined on the basisof the distances between the device 16 and the users. For example, theweighting coefficient may increase as the distance between the device 16and the user becomes smaller, and decrease as the distance between thedevice 16 and the user becomes larger. A correlation between theweighting coefficient and the distance may be reversed, instead. Thedistances are calculated on the basis of the positional informationregarding the device 16 and the users.

When Biological Information has Changed

When the brain waves of users A, B, and C have changed, the processor 26of the information processing apparatus 10 changes the operation to beperformed on the device 16 in accordance with the change. For example,the processor 26 calculates an average of the brain waves of users A, B,and C at predetermined time intervals, identifies an operation at thetime intervals on the basis of the average, and operates the device 16at the time intervals in accordance with the identified operation. Inthis case, too, the above-described weighting may be performed. In thefirst example, the plural living things whose biological information isto be measured may be a combination of at least two of a person, ananimal other than a person, and a plant, instead. For example, thedevice 16 may be operated on the basis of biological informationregarding a person and a living thing. The same holds in the followingexamples.

Process to be Performed when Living Thing has Moved

A process to be performed when a living thing has moved will bedescribed with reference to FIG. 9. FIG. 9 schematically illustrates theroom 36. As in the example illustrated in FIG. 8, there are users A, B,and C, who are the persons, in the room 36. The device 16 is alsoprovided in the room 36.

If user A has moved as indicated by an arrow in FIG. 9, the distancebetween user A and the device 16 exceeds the threshold whereas thedistances between users B and C and the device 16 are smaller than orequal to the threshold, for example, the processor 26 of the informationprocessing apparatus 10 identifies an operation on the basis of thebrain waves of users B and C and operates the device 16 in accordancewith the identified operation. As described above, the processor 26 mayidentify an operation on the basis of an average of the brain waves ofusers B and C and operate the device 16 in accordance with theidentified operation, identify function levels on the basis of the brainwaves of users B and C and operate the device 16 in accordance with anaverage of the identified function levels, or determine turning on oroff on the basis of the brain waves of users B and C and turn on or offthe device 16 in accordance with a result of the determination.

The processor 26 may perform the above-described weighting. If user Abefore the movement is a user who affects a determination of anoperation to be performed on the device 16 in this case (e.g., if theweighting coefficient of user A is larger than those of the otherusers), the operation might change as a result of the movement of userA.

In the case of measurement of biological information regarding animalsand living things other than persons, too, the operation to be performedon the device 16 might change if an animal moves or a position of aliving thing changes.

Second Example

A second example will be described hereinafter. In the second example,biological information measured in plural living things located atdifferent places is used. An example will be described hereinafter inwhich the device 16 is operated on the basis of biological informationmeasured in persons located at different places (e.g., places indifferent rooms). A scene where the second example is applied, forexample, is a scene where plural persons have a conversation or ameeting online (e.g., a remote conference, a web conference, etc.).

It is assumed, for example, that users A, B, C, and D, which arepersons, are attending an online meeting. User A is using a terminalapparatus 18A, user B is using a terminal apparatus 18B, user C is usinga terminal apparatus 18C, and user D is using a terminal apparatus 18D.The terminal apparatuses 18A to 18D have the same configuration as thatof the terminal apparatus 18. For example, the terminal apparatuses 18Ato 18D can receive a service for having an online conversation ormeeting from a server that provides the service and communicateinformation (e.g., images, sounds, text, etc.) with one another. A knownservice, for example, is used as the service.

The second example will be described in detail hereinafter withreference to FIG. 10. FIG. 10 illustrates a screen 38A. The screen 38Ais displayed on the display of the UI 30 of the terminal apparatus 18Aused by user A.

Images of users A to D, who are attending the online meeting, aredisplayed on the screen 38A. For example, the screen 38A includes areas40A, 40B, 40C, and 40D. The images of users A to D are displayed on theareas 40A to 40D, respectively. Screens having the same configuration asthat of the screen 38A are also displayed on the displays of theterminal apparatus 18B to 18D, respectively.

For example, a camera provided for the terminal apparatus 18A used byuser A or a camera that is not provided for the terminal apparatus 18Acaptures an image of user A, and the processor 34 of the terminalapparatus 18A displays the captured image in the area 40A. The terminalapparatus 18A transmits data regarding the image of user A to theterminal apparatuses 18B to 18D, and displays of the terminalapparatuses 18B to 18D display the image of user A. The same holds forimages of users B to D.

As in the first example, the biological information measuringapparatuses 12 measure biological information regarding the users. It isassumed here, for example, that brain waves are measured in users A to Das in the first example. Biological information other than brain wavesmay be measured, instead. The biological information measuringapparatuses 12 used by the users transmit biological information (e.g.,brain wave signals) regarding users A to D, respectively, to theinformation processing apparatus 10.

The processor 26 of the information processing apparatus 10 operates thedevice 16 on the basis of the brain waves of users A to D. The terminalapparatuses 18A to 18D may be the devices 16 to be operated, or a deviceother than the terminal apparatuses 18A to 18D may be the device 16 tobe operated.

The terminal apparatuses 18 may each have the function of theinformation processing apparatus 10, and the processor 34 of each of theterminal apparatuses 18 may operate the device 16 to be operated. Forexample, each of the terminal apparatuses 18 receives biologicalinformation regarding each of the users from the correspondingbiological information measuring apparatus 12 through a communicationpath and operates the device 16.

For example, the processor 34 of each of the terminal apparatuses 18controls sound volume of a speaker provided for the terminal apparatus18 on the basis of brain waves of the corresponding user. If it isdetermined as a result of an analysis of brain waves of user B that userB is excited, for example, the processor 34 of each of the terminalapparatuses 18 controls the sound volume of the speaker provided for theterminal apparatus 18 such that sound volume of user B becomes lower.For example, the processor 34 of the terminal apparatus 18A controls thesound volume of the speaker provided for the terminal apparatus 18A suchthat the sound volume of user B becomes lower. The same holds for theother terminal apparatuses 18. When a speaker that is not provided forthe terminal apparatuses 18 is used, the processor 34 controls soundvolume of the speaker.

Users who are going to attend an online meeting may be authenticated.Only authenticated users may attend an online meeting. For example, arecognition technique such as face recognition, voice recognition,retina recognition, or otoacoustic emission recognition may be used. Theinformation processing apparatus 10 or an authentication server mayperform the recognition.

In the second example, too, the processor 26 of the informationprocessing apparatus 10 may operate the device 16 on the basis of brainwaves of users (e.g., on the basis of an average of the brain waves).The device 16 to be operated may be a device used by at least one ofusers A to D, or another device. In this case, weighting may beperformed.

Process to be Performed when Living Thing has Moved

A process to be performed when a living thing has moved will bedescribed hereinafter with reference to FIG. 11. FIG. 11 illustrates thescreen 38A.

If user B who has been attending an online meeting leaves or the imageof user B is no longer displayed in the area 40B, for example, theprocessor 26 of the information processing apparatus 10 operates thedevice 16 on the basis of the brain waves of users A, C, and D.

Third Example

A third example will be described hereinafter with reference to FIG. 12.The third example is a combination of the first and second examples.FIG. 12 illustrates the screen 38A.

It is assumed, for example, that users A1 and A2 are attending an onlinemeeting using the terminal apparatus 18A, users B1 and B2 are attendingthe online meeting using the terminal apparatus 18B, user C1 isattending the online meeting using the terminal apparatus 18C, and usersD1, D2, and D3 are attending the online meeting using the terminalapparatus 18D. For example, users A1 and A2 belong to group A, users B1and B2 belong to group B, user C1 belongs to group C, and users D1, D2,and D3 belong to group D. Information regarding the groups (e.g.,identification information regarding the groups, identificationinformation regarding the users belonging to the groups, etc.) is storedin the information processing apparatus 10, a server, or the like.

Images of users A1 and A2 are displayed in the area 40A, images of usersB1 and B2 are displayed in the area 40B, an image of user C1 isdisplayed in the area 40C, and images of users D1, D2, and D3 aredisplayed in the area 40D.

As in the first example, the biological information measuringapparatuses 12 measure biological information regarding the users.Biological information regarding one or more users attending an onlinemeeting need not be measured. It is assumed here, for example, thatbrain waves are measured in the users as in the first example.Biological information other than brain waves may be measured, instead.The biological information measuring apparatuses 12 used by the userstransmit biological information (e.g., brain wave signals) regarding theusers to the information processing apparatus 10.

As in the second example, the processor 26 of the information processingapparatus 10 operates the device 16 on the basis of the brain waves ofthe users. The device 16 to be operated may be the terminal apparatuses18A to 18D or a device other than the terminal apparatuses 18A to 18D,instead. Each of the terminal apparatuses 18A to 18D may have thefunction of the information processing apparatus 10, and the processor34 of each of the terminal apparatuses 18A to 18D may operate the device16 to be operated.

As in the second example, for example, the processor 34 of each of theterminal apparatuses 18 controls the sound volume of the speakerprovided for the terminal apparatus 18. If it is determined as a resultof an analysis of the brain waves of user B that user B is excited, forexample, the processor 34 of each of the terminal apparatuses 18controls the sound volume of the speaker provided for the terminalapparatus 18 such that the sound volume of user B becomes lower.

In another example, the processor 34 of each of the terminal apparatuses18 may calculate an average of brain waves of one or more usersbelonging to a group and determine a state of the group on the basis ofthe average. If it is determined as a result of an analysis of anaverage of brain waves of users B1 and B2 belonging to group B thatusers B1 and B2 are excited, for example, the processor 34 of each ofthe terminal apparatuses 18 controls the sound volume of the speakerprovided for the terminal apparatus 18 such that sound volume of usersB1 and B2 becomes lower.

In the third example, too, the processor 26 of the informationprocessing apparatus 10 may operate the device 16 on the basis of brainwaves of users (e.g., on the basis of an average of the brain waves).The processor 26 may operate the device 16 on the basis of brain wavesof one or more users belonging to a group (e.g., on the basis of anaverage of the brain waves of the one or more users belonging to thegroup), instead.

Process to be Performed when Living Thing has Moved

A process to be performed when a living thing has moved will bedescribed hereinafter with reference to FIG. 13. FIG. 13 illustrates thescreen 38A.

If user C1 who has been attending an online meeting leaves or an imageof user C1 is no longer displayed in the area 40C, for example, theprocessor 26 of the information processing apparatus 10 operates thedevice 16 on the basis of brain waves of the users belonging groups A,B, and D.

Fourth Example

A fourth example will be described hereinafter. In the fourth example, aprocess for encouraging a person to move is performed.

For example, the processor 26 of the information processing apparatus 10obtains emotion information, mental information, or psychologicalinformation regarding a person by analyzing biological informationregarding the person. When plural types of biological information aremeasured, the processor 26 may obtain emotion information, mentalinformation, or psychological information regarding a person byanalyzing some or all of the plural types of biological information. Theprocessor 26 may weigh the types of biological information on the basisof order of priority of the types of biological information and obtainemotion information, mental information, or psychological informationregarding the person on the basis of a result of the weighting, instead.Alternatively, a reference level may be set for each of the types ofbiological information, and the processor 26 may determine, for types ofbiological information whose reference levels are the same, thatemotions, mental states, or psychological states have the same tendency.

The processor 26 encourages the person to go to an optimal place on thebasis of a result of an analysis of biological information. Ifdetermining on the basis of the result of the analysis of the biologicalinformation that the person is feeling uncomfortable, for example, theprocessor 26 encourages the person to go to a place where the person islikely to feel comfortable. The place where the person is likely to feelcomfortable is an example of the optimal place.

The processor 26 identifies an optimal place (e.g., a place where aperson is likely to feel comfortable) on the basis of biologicalinformation measured by the biological information measuring apparatus12 and environment information measured by the environment informationmeasuring apparatuses 14.

A specific example will be described. If determining on the basis ofbiological information regarding a person that the person is feelinghot, the processor 26 encourages the person to go to a cooler place. Thecooler place is an example of the optimal place. For example, theprocessor 26 analyzes the amount of sweat and brain waves of a person.If the amount of sweat is larger than or equal to a threshold and thebrain waves indicate discomfort, for example, the processor 26determines that the person is feeling hot. For example, the environmentinformation measuring apparatuses 14 measure environment informationsuch as temperature, humidity, and wind volume in different places andtransmit the environment information regarding the places to theinformation processing apparatus 10. The processor 26 identifies a placewhere the person is likely to feel cool on the basis of the environmentinformation regarding the places. For example, the processor 26identifies a place where temperature is lower than in a current locationof the person and encourages the person to go to the identified place.

If determining on the basis of biological information regarding a personthat the person is feeling cold, the processor 26 encourages the personto go to a warmer place. In this case, the warmer place is an example ofthe optimal place.

For example, the processor 26 encourages each person to go to optimalplaces. When there are users A, B, and C in the room 36 as illustratedin FIG. 8, for example, the processor 26 encourages users A, B, and C togo to their respective optimal places. The processor 26 may guide userswhose biological information has the same tendency to the same place. Ifusers A and B are feeling hot and user C is feeling cold, for example,the processor 26 guides users A and B to a cooler place and user C to awarmer place.

In another example, when a person is looking at an image or textdisplayed on a display, the processor 26 encourages a person to movetoward the display if determining on the basis of biological informationregarding the person that the person is feeling that the image or thetext is hard to see.

In yet another example, when a speaker is outputting a sound, theprocessor 26 encourages a person to move toward the speaker ifdetermining on the basis of biological information regarding the personthat the person is feeling that the sound is too small.

For example, the processor 26 transmits information for encouragingmovement (e.g., information indicating guidance or leading etc.) to theterminal apparatus 18 owned by the person. The processor 34 of theterminal apparatus 18 displays the information on the display of the UI30. FIG. 14 illustrates an example of the information. A screen 42 isdisplayed on the display of the terminal apparatus 18. The screen 42includes a message such as “Go to XX (comfortable place)” and a map 44indicating the comfortable place. A mark 46 indicating a currentposition of the person, a mark 48 indicating the comfortable place, andan arrow 50 for guiding the person to the comfortable place, forexample, are displayed on the map 44. The current position of the personand the comfortable place are identified using a technique such as aGPS. In the example illustrated in FIG. 8, information for encouragingmovement is displayed on the displays of the terminal apparatuses 18owned by users A, B, and C.

FIG. 15 illustrates another example of the information. FIG. 15illustrates the room 36. For example, the processor 26 of theinformation processing apparatus 10 may, as indicated by an arrow 52,display guidance by radiating light onto a floor of the room 36. Forexample, the processor 26 displays the guidance by controlling alighting device provided inside the room 36. When a person is on a roador the like, the processor 26 displays the guidance on a surface of theroad. Alternatively, the processor 26 may display the guidance on a wallor guide a person by sound. The terminal apparatus 18 may output thesound, or another speaker may output the sound. Alternatively, theprocessor 26 may cause a display provided near a person to display theguidance.

The processor 26 may guide persons whose biological informationindicates the same tendency to the same place and persons whosebiological information indicates different tendencies to differentplaces. For example, the processor 26 may guide persons who like eachother to the same place and persons who dislike each other to differentplaces.

Fifth Example

A fifth example will be described hereinafter. In the fifth example,biological information regarding living things is displayed. The fifthexample will be described in detail with reference to FIG. 16. FIG. 16illustrates the screen 38A. It is assumed here, for example, that theusers are holding an online meeting as in the third example.

For example, biological information regarding each of the users istransmitted to the terminal apparatus 18 owned by the user, and theprocessor 34 of the terminal apparatus 18 displays the biologicalinformation regarding the user on the display. FIG. 16 illustrates thescreen 38A of the terminal apparatus 18A owned by user A, and theprocessor 34 of the terminal apparatus 18A displays the biologicalinformation regarding the users on the screen 38A. FIG. 16 illustratesbrain waves, which are an example of the biological information.Waveforms of the brain waves, for example, are displayed.

In the example illustrated in FIG. 16, brain waves 54 of user A1, brainwaves 56 of user B1, brain waves 58 of user C1, and brain waves 60 ofuser D3 are displayed. For example, the processor 34 of the terminalapparatus 18A displays the brain wave 54 of user A1 in the area 40Awhile associating the brain waves 54 with an image of user A1. The sameholds for the brain waves of the other users.

The users can determine whether to display brain waves. The processor 34of the terminal apparatus 18A displays or does not display brain wavesof a user in accordance with an instruction made by user A of theterminal apparatus 18A or an instruction made by the user whose brainwaves have been measured.

If user A of the terminal apparatus 18A determines that brain waves ofusers A1, B1, C1, and D3 are to be displayed, the processor 34 of theterminal apparatus 18A displays the brain waves of users A1, B1, C1, andD3 as illustrated in FIG. 16.

In another example, user A1 may determine whether to display the brainwaves thereof. If user A1 determines that the brain waves thereof are tobe displayed, the brain waves thereof are displayed. If user A1determines that the brain waves thereof are not to be displayed, thebrain waves thereof are not displayed. The same holds for the otherusers.

The processor 34 of the terminal apparatus 18 may display, in additionto a waveform of brain waves, information indicating an emotion, amental state, or psychological state obtained by analyzing the brainwaves. The brain waves 58, for example, indicate a state of arousal(excitement), and the processor 34 displays text indicating the arousal(excitement) in the area 40C along with the brain waves 58. The sameholds for the other brain waves.

The processor 26 of the information processing apparatus 10 may convertsound data during the online meeting into text data and store the textdata in the memory while associating the text data with biologicalinformation (e.g., brain waves) regarding the users. In addition, theprocessor 26 may store the sound data in the memory while associatingthe sound data with the biological information regarding the users. Inaddition, the processor 34 of the terminal apparatus 18 may obtain thetext data and display text indicated by the text data. For example,sound data or text data associated with biological information may besearched for using the biological information as a search key.

The processor 34 of the terminal apparatus 18 may display brain wavesindicating a predetermined emotion, mental state, psychological state,or intention information. If brain waves that are not suitable for ameeting (e.g., brain waves indicating sleepiness, sleep, etc.) aremeasured, for example, the processor 34 displays the brain waves whileassociating the brain waves with an image of a user in which the brainwaves have been measured.

Images indicating emotions, mental states, psychological states, orintention information may be displayed. FIG. 17 illustrates an exampleof the images. An image 62 indicating concentration, an image 64indicating sleepiness, and an image 66 indicating relaxation, forexample, are displayed. For example, the processor 34 of the terminalapparatus 18 displays the images in addition to, or instead of, brainwaves while associating the image with the users.

The processor 34 of the terminal apparatus 18 may change the number ofimages, color, a highlighting method, a type of images, or the like inaccordance with a degree of an emotion, a mental state, a psychologicalstate, or intention information.

The processor 34 may display a score according to a degree of anemotion, a mental state, a psychological state, or intention informationwhile associating the score with a user.

The processor 34 may process and display an image of a user (e.g., aface image of the user) or change a background image of the user inaccordance with a degree of an emotion, a mental state, a psychologicalstate, or intention information. If a user has a negative feeling, forexample, the processor 34 displays the user's face in white or a palecolor. If a user has a positive feeling, the processor 34 displays theuser's face in a bright color. Alternatively, if a user has a negativefeeling, the processor 34 may display a background image in a darkertone. If a user has a positive feeling, the processor 34 may display abackground image in a lighter tone.

Sixth Example

A sixth example will be described hereinafter with reference to FIG. 18.FIG. 18 schematically illustrates tomographic images of a person'sbrain.

For example, the biological information measuring apparatus 12 measuresbrain waves of a person, and the processor 26 of the informationprocessing apparatus 10 identifies active parts of the person's brain onthe basis of the measured brain waves. For example, the processor 26measures brain waves with three or more electrodes attached to theperson's head and identifies parts where the measured brain waves arecaused. The processor 26 may display marks indicating the identifiedparts on tomographic images of the person's brain or apply a color tothe identified parts.

The tomographic images may be ones captured by a computed tomography(CT) apparatus or a magnetic resonance imaging (MRI) apparatus or may beimages schematically expressing the brain (e.g., prepared images).

Seventh Example

A screen on which biological information is displayed will be describedhereinafter. FIG. 19 illustrates an example of a home screen.

A home screen 100 is displayed, for example, on the display of theterminal apparatus 18. A “Scan” button, a “Record” button, a “Reset”button, a “File” button, and an “Edit” button are displayed on the homescreen 100. The home screen 100 also includes display areas 102, 104,and 106. In the example illustrated in FIG. 19, the areas 102, 104, and106 are arranged from top to bottom in this order, but this arrangementis just an example. The user may change the arrangement.

When the “Scan” button is pressed, a list of biological informationmeasuring apparatuses 12 capable of communicating (e.g., can be pairedusing Bluetooth (registered trademark) low energy (BLE)) with theterminal apparatus 18. When the “Record” button is pressed, biologicalinformation measured by the biological information measuring apparatuses12 is stored in the memory. For example, the biological informationmeasuring apparatuses 12 and the terminal apparatus 18 communicate witheach other, and the biological information measured by the biologicalinformation measuring apparatuses 12 is transmitted from the biologicalinformation measuring apparatuses 12 to the terminal apparatus 18 andstored in the memory 32 of the terminal apparatus 18. When the “Reset”button is pressed, calibration settings are reset. When the “File”button is pressed, a screen for managing files of biological informationis displayed. When the “Edit” button is pressed, a setting screen (referto FIG. 23) that will be described later is displayed.

Biological information measured by the biological information measuringapparatuses 12 is displayed in the area 102. Information obtained byprocessing the biological information is displayed in the area 104.Unprocessed biological information will be referred to as “raw data”hereinafter. For example, a graph of raw data (hereinafter referred toas a “first graph”) is displayed in the area 102, and a graph ofprocessed biological information (hereinafter referred to as a “secondgraph”) is displayed in the area 104. Setting values and the like of thegraph displayed in the area 104 are displayed in the area 106.

A vertical axis (Y-axis) of the first graph represents potential (e.g.,[V]) indicating biological information, and a horizontal axis (X-axis)represents time. The range of the vertical axis can be changed, forexample, through pinching, zooming, or the like.

A vertical axis (Y-axis) of the second graph before switching representspotential (e.g., [V]) indicating biological information, and ahorizontal axis (X-axis) represents time. The range of the vertical axiscan be changed, for example, through pinching, zooming, or the like.

The information displayed in the areas 104 and 106 may be switched toother pieces of information. The information is switched, for example,when an operation for sliding the area 104 or 106 such as a flick. Theinformation may be switched through another operation, instead.

FIG. 20 illustrates information displayed as a result of the switching.

The second graph before the switching is, for example, a graph (e.g., areal-time graph) generated by performing a process set in the settingscreen, which will be described later, on raw data. Setting values andthe like relating to the second graph are displayed in the area 106before the switching. An index indicating that biological information(e.g., brain waves) has been obtained, an index indicating that a personin which biological information has been measured is concentrating, anindex indicating that the person in which biological information hasbeen measured is relaxed, values of a six-axis sensor (e.g., valuesindicating the amount of movement in X, Y, and Z-axis directions, valuesindicating the amount of rotation about X, Y, and Z-axes, etc.) mountedon the biological information measuring apparatus 12, for example, aredisplayed in the area 106. The values displayed in the area 106 areupdated, for example, at predetermined time intervals (e.g., everysecond). An image having a color corresponding to each value may also bedisplayed.

The second graph after the switching is a graph (hereinafter referred toas an “FFT graph”) generated by performing an FFT on raw data. Settingvalues relating to the first and second graphs and the FFT graph aredisplayed in the area 106 after the switching. An item “Graph X”, anitem “Graph Y”, an item “FFT Max X”, and an item “FFT Max Y”, forexample, are displayed in the area 106. The item “Graph X” is used toset a value common to the X-axes of the first graph and the FFT graph.The item “Graph Y” is used to set values of the Y-axes of the first andsecond graphs. An item “Graph1” is used to set a value of the Y-axis ofthe first graph, and an item “Graph2” is used to set a value of theY-axis of the second graph. The item “FFT Max X” is used to set amaximum value of the X-axis of the FFT graph. The item “FFT Max Y” isused to set a maximum value of the Y-axis of the FFT graph.

FIG. 21 illustrates a specific example of the first graph of raw dataand the second graph before the switching. FIG. 22 illustrates aspecific example of the second graph after the switching (the graphsubjected an FFT).

FIG. 23 illustrates an example of the setting screen. When the “Edit”button is pressed on the home screen 100, a setting screen 110illustrated in FIG. 23 is displayed.

An item “Micro Volts” is used to set “V” as a unit for the Y-axes of thefirst and second graphs.

An item “Cut Off Filter” is used to set setting values of filters (e.g.,a high-pass filter (HPF) and a low-pass filter (LPF)) used forbiological information. The biological information is filtered inaccordance with these setting values. For example, the first and secondgraphs are filtered. The filtering may be performed by the biologicalinformation measuring apparatus 12, the information processing apparatus10, or the terminal apparatus 18. As a result of the filtering,information having certain frequencies can be extracted from themeasured biological information. For example, only brain waves ormyoelectric signals (e.g., signals based on movement of the eyes, theface, the jaws, and the like) can be extracted from biologicalinformation including the brain waves and the myoelectric signals. Thedevice 16 may be operated on the basis of the extracted signals. Forexample, the processor 26 of the information processing apparatus 10 mayseparately extract the brain waves and the myoelectric signals andoperate the same device 16 or different devices 16 on the basis of thebrain waves and the myoelectric signals.

An item “Peak Filter” is used to set values relating to processing for apeak value of raw data. The item “Peak Filter” includes, for example, anitem for setting an upper limit to the peak value of raw data inmeasurement of biological information (e.g., brain waves) and an itemfor setting a value for measuring impulse noise.

Eighth Example

An eighth example will be described hereinafter. In the eighth example,the processor 26 of the information processing apparatus 10 estimates,on the basis of biological information regarding a person, an emotion, amental state, or a psychological state of another person having acertain relationship with the foregoing person. The processor 26 mayestimate, on the basis of emotion information, mental information, orpsychological information regarding a person, an emotion, a mentalstate, or a psychological state of another person, instead.

The certain relationship is, for example, a parent-child relationship ora blood relationship. For example, the processor 26 may estimate anemotion, a mental state, or a psychological state of a child on thebasis of biological information, emotion information, mentalinformation, or psychological information regarding a parent, anemotion, a mental state, or a psychological state of a parent on thebasis of biological information, emotion information, mentalinformation, or psychological information regarding a child, or anemotion, a mental state, or a psychological state of a person on thebasis of biological information, emotion information, mentalinformation, or psychological information regarding the person'ssibling.

For example, the biological information measuring apparatuses 12 measurebiological information regarding a parent and biological informationregarding a child. If the biological information regarding the child issimilar to that regarding the parent (e.g., if brain waves of the childhave a waveform similar to a waveform of brain waves of the parent), theprocessor 26 of the information processing apparatus 10 estimates thatan emotion, a mental state, or a psychological state of the child is thesame as an emotion, a mental state, or a psychological state of theparent. If the biological information regarding the parent indicatesthat the parent is feeling good and the biological information regardingthe child is similar to that regarding the parent, for example, theprocessor 26 estimates that the child is also feeling good. Such anestimation is performed, for example, when the child is a baby, aninfant, or the like for whom it is difficult to control his/her emotion,mental state, or psychological state.

In another example, when the parent is a person for whom it is difficultto control his/her emotion, mental state, or psychological state (e.g.,a dementia sufferer), the processor 26 may estimate an emotion, a mentalstate, or a psychological state of the parent on the basis of biologicalinformation, emotion information, mental information, or psychologicalinformation regarding the child.

In yet another example, the processor 26 may estimate emotions, mentalstates, or psychological states of persons whose lifestyles are similarto each other. Persons who have been living together for a long time,for example, often have similar lifestyles. In this case, the processor26 estimates an emotion, a mental state, or a psychological state of aperson on the basis of biological information, emotion information,mental information, or psychological information regarding anotherperson who has a lifestyle similar to the foregoing person.

Ninth Example

A ninth example will be described hereinafter. In the ninth example, theprocessor 26 of the information processing apparatus 10 notifies atarget, who is a person, of certain information that the target does notknow. For example, the processor 26 transmits the certain information toa terminal apparatus 18 owned by the target.

The certain information of which the target is notified is, for example,visual information, environment information, atmosphere informationregarding a certain place, or the like. Information that can be obtainedonly in the certain place, for example, is the information of which thetarget is to be notified. The certain place is specified, for example,by a manager of the information processing apparatus 10, the target, orthe like. In addition, information regarding a danger caused in thecertain place or information indicating a danger that can be caused inthe certain place may be transmitted to the terminal apparatus 18 ownedby the target in accordance with an instruction from the manager, thetarget, or the like.

The visual information is, for example, an image captured by a fixedcamera provided in the certain place, an image captured by a camera wornby a person in the certain place, or the like. The visual informationmay include sound information measured by a microphone provided in thecertain place or a microphone worn by a person in the certain place. Theenvironment information is information (e.g., temperature, humidity, airquality, weather, etc.) measured by various sensors provided in thecertain place or information measured by a sensor worn by a person inthe certain place. The atmosphere information is, for example,information indicating that the certain place is comfortable oruncomfortable to a person, information indicating whether the certainplace is a place where a person can concentrate, or the like. If aperson in the certain place inputs atmosphere information regarding thecertain place to a terminal apparatus 18 owned thereby, for example, theterminal apparatus 18 transmits the input information to the informationprocessing apparatus 10.

If the target specifies a certain place and makes a request to obtaincertain information regarding the certain place using the terminalapparatus 18, for example, the terminal apparatus 18 transmitsinformation indicating the request to the information processingapparatus 10. The processor 26 of the information processing apparatus10 transmits the certain information regarding the certain place to theterminal apparatus 18 in response to the request. The processor 34 ofthe terminal apparatus 18 displays the certain information on thedisplay or outputs the certain information from the speaker as a sound.

The processor 26 of the information processing apparatus 10 may observesituations in various places and, if a situation in a certain placespecified by the target satisfies a predetermined condition, transmitcertain information regarding the certain place to the terminalapparatus 18 owned by the target. A situation in a place is, forexample, a situation relating to an environment obtained by analyzingenvironment information (e.g., temperature, humidity, weather, etc.), asituation obtained from visual information (e.g., crowdedness of a town,a train, etc.), or a dangerous situation obtained from visualinformation, environment information, or the like. A situation in aplace may be estimated or predicted from visual information, environmentinformation, or the like.

If weather in a certain place specified by the target is, or ispredicted to become, predetermined weather, for example, the processor26 of the information processing apparatus 10 transmits informationregarding the weather in the certain place to the terminal apparatus 18owned by the target. If crowdedness of a certain place is, or ispredicted to become, a predetermined level of crowdedness, the processor26 of the information processing apparatus 10 transmits informationregarding the crowdedness of the certain place to the terminal apparatus18 owned by the target.

The processor 26 of the information processing apparatus 10 may selectcertain information to be transmitted to the terminal apparatus 18 ownedby the target on the basis of a purpose of the target (e.g., to avoiddangers etc.). The purpose is specified, for example, by the target.

If an automobile is out of control and a person gives, using a terminalapparatus 18, an instruction to transmit information regarding theautomobile to the information processing apparatus 10, for example, theterminal apparatus 18 transmits the information regarding the automobile(e.g., information including information indicating a danger) to theinformation processing apparatus 10. The information regarding theautomobile includes positional information regarding the terminalapparatus 18. If a camera worn by the person captures an image of theautomobile, the information regarding the automobile may include thecaptured image. Upon receiving the information regarding the automobile,the processor 26 of the information processing apparatus 10 obtains animage captured by a fixed camera provided near a position of theterminal apparatus 18 (e.g., an image captured at a higher position)from the fixed camera and transmits the image to the terminal apparatus18 owned by the target. The image is displayed on the display of theterminal apparatus 18 owned by the target. The processor 26 may alsotransmit the information indicating the danger to the terminal apparatus18 owned by the target. The information indicating the danger isdisplayed on the display of the terminal apparatus 18 owned by thetarget. When plural fixed cameras are provided, the processor 26 maytransmit, to the terminal apparatus 18 owned by the target, an imagethat has been captured by one of the fixed cameras and that is the mosteffective in detecting the danger.

When transmitting the certain information to the terminal apparatus 18owned by the target, the processor 26 may select information to beincluded in the certain information or process the certain informationin accordance with an attribute (e.g., age, gender, a medical history,etc.) of the target. Since visual abilities can be different dependingon age (e.g., due to cataract, glaucoma, etc.), for example, theprocessor 26 may adjust sharpness of an image included in the certaininformation or the like in accordance with age of the target. Inaddition, since awareness of danger can be different depending on genderand age, the processor 26 may select information to be included in thecertain information in accordance with the gender and the age of thetarget. For example, the processor 26 may include only an image in thecertain information, add text (e.g., text indicating a danger) to thecertain information, or include both an image and text in the certaininformation in accordance with the gender and the age of the target.

Tenth Example

A tenth example will be described hereinafter.

For example, the biological information measuring apparatus 12 maymeasure biological information regarding a plant, and the device 16 maybe operated on the basis of the measured biological information. Thebiological information to be measured is, for example, potential (e.g.,potential including membrane potential etc.) or water content of theplant. Another piece of information may be measured as the biologicalinformation, instead.

In a case where biological information regarding a plant is beingmeasured, for example, if the plant grows more rapidly by playing musicnear the plant than when music is not played, the processor 26 of theinformation processing apparatus 10 causes the device 16 to play musicsuch that biological information (e.g., a potential) measured whilemusic was being played continues to be measured. Alternatively, theprocessor 26 may control a method for giving water and fertilizers tothe plant. For example, the processor 26 may give water and fertilizersto plant such that the biological information continues to be measuredin the plant.

In addition, the processor 26 may detect changes in a state of the planton the basis of the biological information measured in the plant anddisplay information (e.g., an image, text, etc.) according to thechanges on a display of an apparatus such as the terminal apparatus 18.The processor 26 may display the information according to the changes onthe display while personifying the plant, instead.

The processor 26 may determine, on the basis of the biologicalinformation measured in the plant, whether the plant has a disease anddisplay a result of the determination on a display of an apparatus suchas the terminal apparatus 18.

Water content of a living thing changes over time, and biopotentialreflects the changes. The biological information measuring apparatus 12may measure the biopotential, and the processor 26 of the informationprocessing apparatus 10 may detect an action or an abnormality of theliving thing on the basis of the measured biopotential. If anabnormality is detected, for example, the processor 26 may transmitwarning information indicating the abnormality to an apparatus such asthe terminal apparatus 18. If biopotential of a person is measured andan abnormality is detected in the person on the basis of thebiopotential, for example, warning information is transmitted to theterminal apparatus 18 owned by the person and displayed on the displayof the terminal apparatus 18.

In general, water tends to be distributed evenly over the entirety of aperson's body in the morning and gradually accumulates in a lower partof the person's body during the day as the person moves around.Biopotential reflects such changes in water content. The processor 26 ofthe information processing apparatus 10 may measure biopotentials atvarious positions on a person' body and measure time on the basis ofchanges in biopotential. For example, the changes in the biopotentialmay be used as a clock or a timer.

In addition, the processor 26 of the information processing apparatus 10may measure biological information (e.g., brain waves etc.) regarding ananimal other than a person and estimate a state (e.g., a disease) of theanimal or an environment in which the animal exists.

The processor 26 of the information processing apparatus 10 may detect atype of living thing that is wearing the biological informationmeasuring apparatus 12.

The size of a skull, for example, differs between a person and an animalother than a person. When the biological information measuring apparatus12 includes a band which make the biological information measuringapparatus 12 wearable on the head and is worn on the head using theband, the processor 26 may detect the length of the band using a sensoror the like and detect a type of animal that is wearing the biologicalinformation measuring apparatus 12 on the basis of the length of theband. More specifically, since the size of the skull differs between aperson and a dog, the length of the band accordingly differs. Theprocessor 26 detects the length of the band and determines whether aliving thing that is wearing the biological information measuringapparatus 12 is a person or a dog.

In another example, when the biological information measuring apparatus12 is an apparatus worn on the ears of a living thing, the processor 26may detect the length of hair on the ears on which the biologicalinformation measuring apparatus 12 is worn using a sensor or the likeand detect a type of animal that is wearing the biological informationmeasuring apparatus 12 on the basis of the length of the hair.

In yet another example, since body temperature differs between types ofliving things, the processor 26 may measure body temperature of a livingthing that is wearing the biological information measuring apparatus 12and identify a type of living thing on the basis of the bodytemperature.

In yet another example, since height and weight differ between types ofliving things and a resistance value accordingly differs, the processor26 may identify a type of living thing that is wearing the biologicalinformation measuring apparatus 12 on the basis of the resistance value.

A type of living thing (e.g., a person, a dog, a plant, etc.) may beassociated with biological information measured in a living thing. Thebiological information measuring apparatus 12 may make the association,or the information processing apparatus 10 may make the association. Indoing so, a type of living thing in which certain biological informationhas been measured can be identified. When a type of living thing inwhich biological information has been measured is identified asdescribed above, information indicating the identified type may beassociated with the biological information.

Animals may communicate with each other using biological informationsuch as brain waves. For example, persons, or a person and an animalother than a person (e.g., a dog), may communicate with each other usingbiological information. When a type of living thing in which biologicalinformation is identified as described above, the processor 26 maychange, for example, a method for converting brain waves in thecommunication.

Eleventh Example

An eleventh example will be described hereinafter. In the eleventhexample, a stimulus is given to a living thing, and the processor 26 ofthe information processing apparatus 10 associates stimulus informationindicating the stimulus and biological information measured in theliving thing with each other. The stimulus information and thebiological information associated with each other are stored, forexample, in the memory 24 of the information processing apparatus 10.Plural pieces of biological information may be associated with stimulusinformation, instead. Brain waves, pulse rate, blood pressure,complexion, and body temperature, for example, may be associated withstimulus information.

The stimulus is, for example, an image such as a still image or a movingimage, an environment, a taste, or the like. An image indicating anemotion, an image or an environment that induces fear, or a healingimage or environment, for example, is given as the stimulus. Such animage is, for example, a virtual image based on a virtual reality (VR)technique. An image showing an actual object may be used, instead. Thestimulus may be an electrical stimulus given by an electrode providedfor the biological information measuring apparatus 12, instead.

When a person is looking at an image, for example, the biologicalinformation measuring apparatus 12 measures biological information(e.g., brain waves) in the person and associates the image and thebiological information with each other. The brain waves to be measuredare brain waves emitted from the left brain, brain waves emitted fromthe right brain, or brain waves emitted from both the left and rightbrains.

The processor 26 may associate words (i.e., sound information) utteredby a person while a stimulus is being given to the person with stimulusinformation and biological information. The processor 26 may alsoassociate information indicating a motion of the person with thestimulus information and the biological information.

A virtual image such as a VR image may be a background image captured indaily life of a person, and surrounding sounds may be played back whenthe VR image is displayed.

Alternatively, an image based on past experience of a person in whombiological information has been measured may be used as the VR image.The person may be inquired of the past experience in advance (e.g., aquestionnaire).

If biological information measured in a person who is looking at a VRimage greatly changes, the processor 26 may associate a correspondingpiece of the biological information with the stimulus information as aneffective biological information. If the amount of change in thebiological information becomes larger than or equal to a threshold, forexample, the processor 26 associates a corresponding piece of thebiological information with the stimulus information as effectivebiological information.

Order in which the stimulus information is given to a person may becontrolled. For example, visual information that a person dislikes maybe given to the person, and then audio information, olfactoryinformation, or sensation information that the person likes may be givento the person. The processor 26 may associate biological informationmeasured at this time and a corresponding piece of the stimulusinformation with each other.

The processor 26 may change the stimulus information given to a personin accordance with changes in the measured biological information.

For instance, FIG. 40 illustrates brain waves of a person who is viewinga VR image containing a monster-like creature. Change is observed in thebrain waves between a mental state of nervousness before themonster-like creature appears, and a mental state of surprise when themonster-like creature appears, as shown in FIG. 40. Such change can bedetected and the processor 26 may change the stimulus informationaccordingly.

The processor 26 may change speed at which the VR image is played back,a point of view for the VR image, coloring of the VR image, or imagequality or a focal point of the VR image.

The stimulus information and the biological information associated witheach other may be used for digital marketing (e.g., video production,attraction, a product such as a device, etc.) or digital healthcare(e.g., a service for evaluating a lifestyle on the basis of degrees ofstress and refreshment recorded daily etc.).

Twelfth Example

The biological information measuring apparatus 12 may be shared byplural persons or the like. For example, person A may use the biologicalinformation measuring apparatus 12, and then person B may use thebiological information measuring apparatus 12 that has been used byperson A.

For example, if a living thing (e.g., a person) that is wearing abiological information measuring apparatus 12 of a contact type (i.e.,an apparatus that measures biological information with an electrode or asensor in contact with a living thing such as a person) is replaced byanother living thing, the biological information measuring apparatus 12may output information indicating the change or information forencouraging cleaning of the biological information measuring apparatus12. The outputting of the information refers to, for example, displayingof the information on the display, issuing the information from thespeaker as a sound, or both.

When a microphone is provided for the biological information measuringapparatus 12, for example, the biological information measuringapparatus 12 recognizes a person who is using the biological informationmeasuring apparatus 12 on the basis of the person's voice input from themicrophone and determines whether a person who is using the biologicalinformation measuring apparatus 12 has been replaced by another person.A camera provided for the terminal apparatus 18 or the like may capturean image of a person who is using the biological information measuringapparatus 12, and the biological information measuring apparatus 12 mayrecognize the person on the basis of the captured image.

In another example, when the information processing apparatus 10, thebiological information measuring apparatus 12, the terminal apparatus18, a server, or the like manages an account of a person who is usingthe biological information measuring apparatus 12 and the account hasbeen switched to another account, the biological information measuringapparatus 12 may determine that the person who is using the biologicalinformation measuring apparatus 12 has been replaced by another person.

By outputting information indicating that a person who is using thebiological information measuring apparatus 12 has been replaced byanother person, the person who is going to use the biologicalinformation measuring apparatus 12 recognizes that another person hasbeen using the biological information measuring apparatus 12. As aresult, the person who is going to use the biological informationmeasuring apparatus 12 recognizes, for example, that the biologicalinformation measuring apparatus 12 needs to be cleaned. The same holdswhen information for encouraging cleaning of the biological informationmeasuring apparatus 12 is output.

In addition, a sensor or the like that detects completion of cleaning ofthe biological information measuring apparatus 12, for example, may beused. When a sensor (e.g., a sensor that detects alcohol) that detectscompletion of alcohol cleaning of an electrode or a sensor for measuringbiological information is used and the biological information measuringapparatus 12 is about to be used (e.g., the biological informationmeasuring apparatus 12 is turned on) without the electrode or the sensorcleaned with alcohol, for example, the biological information measuringapparatus 12 outputs information indicating that the biologicalinformation measuring apparatus 12 has not been cleaned. Alternatively,the biological information measuring apparatus 12 may remain turned off.When the biological information measuring apparatus 12 is about to beused with the electrode or the sensor cleaned with alcohol, thebiological information measuring apparatus 12 may or may not outputinformation indicating that the biological information measuringapparatus 12 has been cleaned.

Thirteenth Example

A thirteenth example will be described hereinafter with reference toFIG. 24. FIG. 24 illustrates the screen 38A. The screen 38A is the sameas that in the third example (refer to FIG. 12).

In the thirteenth example, information (e.g., an image of a bar, avalue, etc.) indicating an emotion, an intention, a mental state, apsychological state, or the like is displayed. The processor 26estimates the emotion, the intention, the mental state, or thepsychological state on the basis of biological information regarding auser. For example, a bar or a value indicating a degree ofconcentration, relaxation, or stress is displayed. Bars or values may bedisplayed for all users, or a bar or a value may be displayed only for auser specified by a manager or the like.

As an example, brain waves of a person performing a mental task(completing a white puzzle) in the morning, at noon, and in the evening,are illustrated in FIG. 41. It is found that the length of time forwhich the beta waves and the gamma waves are present shortens towardsthe evening, characterizing the extent of fatigue due to concentration.

As another example, brain waves of a person experiencing a head massageis illustrated in FIG. 42. It is found that rising in theta wave rangeis observed, and then rising in the alpha range occurs as the persondozes off and becomes unconscious, characterizing degrees of relaxationeffects.

As another example, brain waves of a person listening to a music ortaking a supplement is illustrated in FIG. 43. The brain waves aremeasured for four minutes with eyes open for each case of (a) base casewith eyes open, (b) only listening to music with eyes open, (c) onlytaking supplements with eyes open, and (d) both listening to music andtaking supplements. The music used is of the type having 963 Hz. Thesupplement is based on measurement taken at three minutes after takingof five drops (25 mg) of CBD. Fast-Fourier Transform is performed persecond for the frequency ranges of each of the brain wave ranges. It isfound that with (b), (c) or (d), the FFT intensity in the alpha waveband increased, with the most increase being observed in case (d) withincrease of 43% compared to the base case (a) in 9.67 Hz range.

As another example, difference in brain waves between beginning of sleepand as successive sleep states occur is illustrated in FIG. 44. When thesleep begins, theta waves begin to appear. As the sleep stage advances,spindle waves begin to appear. In this way, it is possible tocharacterize the depth of sleep and estimate the time that has passedafter the onset of sleep.

As another example, the user's physiological events, such as desire forexcretion or feeling of itches can be detected. FIG. 45 illustrateschange in brain waves when the user experiences itches.

Characteristics of the brain waves such as the above may be converted toa bar or a value and displayed to the user. In this way, the internalstate of the user may be visualized.

In an example illustrated in FIG. 24, bars and values indicating amental state of user C1 are displayed. A bar 120 is an image indicatinga degree of relaxation of user C1. A bar 122 is an image indicating adegree of stress of user C1. The degree of relaxation is “100”, and thedegree of stress is “30”. The processor 26 calculates these values onthe basis of biological information regarding user C1.

The processor 26 may identify an event that has contributed to a stateof the user (e.g., an emotion, an intention, a mental state, apsychological state, etc.) and calculate a degree of contribution of theevent to the state of the user. For example, the processor 26 identifiesan event that has contributed to stress or recovery from the stress anddisplay information indicating a degree of contribution on the screen38A. In doing so, the user easily understands what kind of event and howmuch the event has affected the state of the user. For example, theprocessor 26 detects occurrence of an event using various sensors andestimates a change in the state of the user at the time of the event andstates of the user before and after the occurrence of the event on thebasis of biological information regarding the user. The processor 26estimates a degree of contribution of the event to the state of theuser, for example, on the basis of the change in the state of the user.

The processor 26 may store, in the memory 24, history informationindicating histories of occurrence of events, biological informationregarding the user before and after the occurrence of the events,changes in the biological information regarding the user, states of theuser before and after the occurrence of the events, changes in the stateof the user. The processor 26 may reproduce a past event on the basis ofthe histories indicated by the history information. For example, theprocessor 26 may record an event that has occurred around a time pointat which the user has gotten relaxed and reproduce the same situation.For example, a target value of relaxation is specified, and theprocessor 26 reproduces a situation where the target value can beachieved. The processor 26 may reproduce the situation by combiningtogether playback of music, display of an image or the like, aninstruction to take an action, and the like. The processor 26 may recordthe histories while obtaining states of brain waves of the user,classifying the states of the brain waves, and associating the states ofthe brain waves with working and non-working hours (e.g., times otherthan time of sleeping). The processor 26 may also insert values and barsindicating states of relaxation and the like into an image of a meetingusing a virtual camera.

Fourteenth Example

A fourteenth example will be described hereinafter with reference toFIGS. 25 and 26. FIG. 25 illustrates, as with FIG. 17, the images ofusers A, B, and C. Information (e.g., brain waves etc.) indicatingstates of the users is also displayed while being associated with theusers. It is assumed here, for example, that users A, B, and C areattending the same online meeting.

If all the users attending the online meeting, or a predetermined numberor more of users, are in the same state (e.g., the same emotion,intention, mental state, or psychological state), for example, theprocessor 26 may display frames of images of the users in the same statein a certain color (e.g., red) or give a certain effect. The states ofthe users are estimated on the basis of biological information regardingthe users.

When users A, B, and C are in the same state (e.g., an aroused state),for example, the processor 26 displays a frame surrounding the images ofusers A, B, and C in a certain color and gives a certain effect (e.g., athicker frame) as illustrated in FIG. 26. In doing so, it can beindicated that users A, B, and C are in the same state. If all usersattending an online meeting are in the aroused state, for example, sucha color and an effect are given to show that the online meeting is in anenthusiastic mode.

As in the fourteenth example, information for collectively expressingstates of plural users attending a meeting may be displayed, instead.

Fifteenth Example

A fifteenth example will be described hereinafter.

Data transmitted from the biological information measuring apparatus 12to another apparatus (e.g., the information processing apparatus 10) maybe data generated by performing processing such as an FFT, instead ofraw data of biological information. In doing so, information that isincluded in raw data but that is not included in data generated byperforming processing such as an FFT is prevented from leaking.

Alternatively, the biological information measuring apparatus 12 mayencrypt information indicating a measured item of biological information(e.g., information indicating a channel or a part) and transmit theencrypted information to another apparatus (e.g., the informationprocessing apparatus 10). In doing so, it becomes difficult to identifythe item indicated by the transmitted biological information, therebyimproving security of the biological information.

When measured biological information or information obtained from themeasured biological information is lent or sold to a user, the processor26 may restrict use of the biological information or the information.The restriction may be achieved, for example, by deleting theinformation when a certain period of time elapses or making the userreturn the information to a provider of the information. A place whereuse of the information is performed, purposes of use of the information,or the like may also be defined.

Sixteenth Example

A sixteenth example will be described hereinafter.

In the sixteenth example, an artificial intelligence (AI) assistant thatoutputs sounds is used. The AI assistant performs processing or operatesdevices, for example, in accordance with biological informationregarding a user or an instruction from the user. The AI assistant maybe incorporated into a terminal apparatus owned by the user or anapparatus other than the terminal apparatus, such as the informationprocessing apparatus 10 or a server.

For example, the processor 26 may change the volume of the sounds outputfrom the AI assistant or speed at which the AI assistant speaks inaccordance with biological information regarding the user.

More specifically, if the biological information regarding the userindicates the user's desire to feel relaxed, the processor 26 changes asound output from the AI assistant to a sound that the user likes (e.g.,a higher tone, a lower tone, speed of speech, etc.) or voice of a familymember of the user.

If the biological information regarding the user indicates that the useris in a hurry, the processor 26 causes the AI assistant to speak fasteror output sounds more promptly.

When the user studies using the sounds output from the AI assistant, theprocessor 26 may control the AI assistant such that the AI assistantoutputs the sounds at a speed or with a volume effective for the user inthe study. An effect of the study by the user is estimated on the basisof biological information regarding the user. For example, the processor26 changes the speed or the volume of the sound output from the AIassistant on the basis of the biological information regarding the user.

If the user is attending an online meeting, a telephone conference, orthe like, the processor 26 may estimate a state of a person with whomthe user is talking on the basis of biological information regarding theperson and change, on the basis of a result of the estimation, a sounduttered by the user to the person such that the sound uttered by theuser becomes easier to understand for the person. If the user is angryand uttering a sound that might scare the person, for example, theprocessor 26 may make the sound gentler or express the user's intentionwith a figure, a sentence, or the like without transmitting the sound tothe person. Such a process may be performed by the AI assistant.

In an online meeting or the like, a user (e.g., a first user) might bewearing a biological information measuring apparatus 12 but another user(e.g., a second user) with whom the first user is talking might not bewearing a biological information measuring apparatus 12. In this case,biological information regarding the first user is obtained, andbiological information regarding the second user is not obtained. Inthis case, too, the biological information regarding the first usermight change due to a change in a facial expression of the second useror a sound uttered by the second user, for example, if the first usercan recognize the facial expression of the second user on a screen. Ifthe biological information regarding the first user changes in thismanner, the processor 26 determines that the biological informationregarding the first user has changed because of the second user. Theprocessor 26 may estimate a state (e.g., an emotion, a mental state, apsychological state, etc.) of the second user on the basis of thedetermination. If the second user have a painful look, for example, thefirst user who is looking at the second user's face might feel sorry.The processor 26 therefore estimates the state of the second user on thebasis of the biological information regarding the first user. The AIassistant may inform the first user of how to deal with the second userin such a state.

When voice is given to a character in an animation or a drama or a movieis dubbed, a user who is authorized to determine a voice to be used maylisten to voices of some voice actors, and the processor 26 maydetermine a voice actor to be employed on the basis of biologicalinformation regarding the user. If biological information indicatingsatisfaction is obtained from the user, for example, the processor 26determines a voice actor who has uttered a corresponding sound as thevoice actor to be employed. If biological information indicating a senseof incongruity or discomfort is obtained from the user, the processor 26may introduce another voice actor to the user.

Seventeenth Example

A seventeenth example will be described hereinafter.

The present exemplary embodiment may be applied to a content servicesuch as a video game. For example, the video game may be provided for auser through a terminal apparatus owned by the user or through anotherdevice (e.g., a gaming device). The video game may be an online game orprovided by executing a program installed on the terminal apparatus orthe gaming device. A storage medium storing a program for the video gamemay be used, and the video game may be provided by executing the programusing the terminal apparatus or the gaming device.

For example, the biological information measuring apparatus 12 measuresbiological information regarding the user while the user is playing thevideo game. The processor 26 may change stories, scenes, sounds, imagesin the video game in accordance with emotions of the user obtained fromthe biological information.

If biological information indicating a level of fear higher than orequal to a predetermined threshold is measured in a user who has troublein playing scary video games or experiencing other scary things, forexample, the processor 26 omits violent scenes, reduces sound volume,simplifies dramatic effects, or makes expressions easier to understand.

If biological information indicating a level of fear higher than orequal to the predetermined threshold is not measured in a user who likesscarier scenes, on the other hand, the processor 26 may change how thevideo game is presented such that more violent scenes and scarier scenesare included.

Tastes of a user are set before the video game starts. The processor 26may change how the video game is presented in accordance with healthproblems and age of the user as well as biological information. Theprocessor 26 may also change how the video game is presented foreducational reasons.

The processor 26 may determine a state of a user at predetermined timeintervals on the basis of biological information regarding the user. Asa result, scenes can be changed in real-time, and the video gameproceeds without causing discomfort or a sense of incongruity in theuser. Scenes that are unchangeable for the sake of stories of the videogame may also be set.

If it is determined that the user is feeling that the video game is toodifficult or too easy, the processor 26 may change a level of difficultyof the video game in accordance with the feeling. For example, theprocessor 26 may change an action level of AI that controls enemies ofthe user or the number of enemies. When a mystery game is being played,the processor 26 may change the number of clues for unpuzzling amystery. Alternatively, the processor 26 may change setting values of acharacter controlled by the user, such as attack power and defensepower.

The processor 26 may notify the user of a change in the level ofdifficulty based on the biological information. In doing so, the userrecognizes, for example, the level of difficulty of the video gamehe/she has cleared. In this case, how long the user has played the videogame, changes in the user's emotion during the play, the level ofdifficulty of the video game, and the like may be displayed usingfigures such as graphs.

If a state that the user desires to achieve is achieved while the useris playing the video game, information indicating the state may bestored in the memory 24 of the information processing apparatus 10 ashistory information. The processor 26 may change scenes or the level ofdifficulty of the video game such that the state that the user desiresto achieve is achieved. If the user sets, to the information processingapparatus 10, information indicating that the user desires to be freedfrom stress and refresh, for example, the processor 26 may change thescenes or the level of difficulty of the video game such that the userachieves the state. For example, the processor 26 may change a scene inthe video game to one including a blue sky and a plain. In anotherexample, the processor 26 may make an enemy weaker so that a charactercontrolled by the user can easily defeat the enemy.

Eighteenth Example

An eighteenth example will be described hereinafter. In the eighteenthexample, the biological information measuring apparatus 12 is a hearableand worn on the user's ears. The hearable may have a sound assistancefunction. The sound assistance function is a function of assisting theuser on the basis of biological information regarding the user. Thesound assistance function may be achieved by AI.

A process in the eighteenth example will be described hereinafter.

First, the hearable, which is an example of the biological informationmeasuring apparatus 12, measures biological information (e.g., anelectrocardiographic waveform, brain waves, a myoelectric waveform, skinpotential, etc.) at the user's ears (e.g., ear canals) and also measuresthe user's voice. The hearable stores the biological information andsound information in chronological order while associating thebiological information and the sound information with each other. Thesepieces of information may be stored in the hearable, a terminalapparatus owned by the user, or the information processing apparatus 10.It is assumed here, for example, that these pieces of information arestored in the information processing apparatus 10.

The processor 26 of the information processing apparatus 10 determineswhether a fluctuation has occurred in the biological informationregarding the user. When the biological information is brain waves, forexample, a fluctuation may be an abnormal epileptic wave, and when thebiological information is an electrocardiographic waveform, afluctuation may be a waveform indicating an abnormality in myocardia,such as arrhythmia. If detecting a fluctuation in the biologicalinformation regarding the user, the processor 26 records a time at whichthe fluctuation has occurred, a place where the user is located at thetime, and information indicating a state of the user (hereinaftercollectively referred to as “context information”). The processor 26 mayremove noise and determine whether the fluctuation derives from a mentalor physical stimulus response.

If the fluctuation (i.e., a change in a mental or physical stimulusresponse) indicates a dangerous condition, the processor 26 controls asound function of the hearable. The dangerous condition is, for example,a case where a symptom of a heart disease has occurred (e.g., whenarrhythmia has been detected from pulse, which is the biologicalinformation), a case where the user has fallen down (e.g., when a fallof the user has been detected on the basis of a myoelectric waveform,which is the biological information, or data from an accelerationsensor), a case where the user has swallowed something (e.g., when anirregular waveform is included in a myoelectric waveform, which is thebiological information, and it has been detected on the basis of theirregular waveform that the user might have swallowed something), or thelike.

The processor 26 transmits the context information, the biologicalinformation, and the sound information to an apparatus (e.g., a PC etc.)provided in a predetermined place (e.g., a hospital etc.). At this time,the processor 26 transmits, to the apparatus, the context information,the biological information, and the sound information measured aroundthe time (e.g., in a predetermined period of time) at which thefluctuation has occurred

The processor 26 may control sounds output from the hearable on thebasis of the context information, the biological information, and thesound information. For example, the processor 26 controls playback speedof content, which is played back by the hearable. More specifically, theprocessor 26 changes speed at which a moving image for learning isplayed back or speed at which an electronic book is read aloud.Alternatively, the processor 26 may stop the playback of the content.Alternatively, the processor 26 may play back content (e.g., a subject,a comic book, a song, etc.) that suits the user's tastes. Alternatively,the processor 26 may execute an automatic call function. The automaticcall function herein refers to a function of automatically calling apredetermined emergency contact number or sending a message to or apredetermined emergency address (e.g., a family member, a regulardoctor, etc.). Calls may be made or messages may be sent to pluralpersons at once or gradually. The processor 26 may change a person to becontacted in accordance with seriousness of a condition of the user. Theuser may also be notified of his/her own condition.

The sound assistance function may be switched for each hearable or useraccount. A function of determining whether to execute the soundassistance function on the basis of biological information may also beprovided. The sound assistance function of the hearable may be executedwithout using biological information. Scene where the sound assistancefunction based on biological information is executed may be limited toscenes where initial symptoms of a disease are examined or a readingfunction for a content service or the like. A function of setting such alimitation may also be provided.

The processor 26 may cause an AI that achieves the sound assistancefunction to learn the context information, the biological information,and the sound information.

For example, the processor 26 may accumulate sound information while theuser is talking in a relaxed state and cause the AI to learn the soundinformation. For example, the AI may create a voice assistant soundoptimized for biological reactions using a technique of sound synthesisor sound adjustment and read the voice assistant sound aloud.

In another example, the processor 26 may accumulate sound informationwhile the user is talking in a stressed state and cause the AI to learnthe sound information. For example, the AI may create a voice assistantsound optimized for recovery of calmness using a technique of soundsynthesis or sound adjustment and read the voice assistant sound aloud.

In yet another example, the processor 26 may accumulate soundinformation while the user is tense and cause the AI to learn when,where, and how the user has become tense. For example, the AI may createa voice assistant sound optimized for recovery of calmness using atechnique of sound synthesis or sound adjustment and read the voiceassistant sound aloud.

In yet another example, the processor 26 may convert the user's voiceinto text, accumulate data regarding the text in chronological orderalong with information indicating the user's emotion, and cause the AIto learn the data and the information. The data and the information maybe stored in a memory, instead. The emotion may be selected by the user,instead. The processor may change the size or color of the text or thelike in accordance with the user's voice. The AI may create a voiceassistant sound optimized for recovery of calmness using a technique ofsound synthesis or sound adjustment and read the voice assistant soundaloud.

For example, the processor 26 stores sound information regarding theuser obtained from a terminal apparatus (e.g., a smartphone, a PC, etc.)owned by the user, information regarding icons, letters, pictures, andthe like input by the user, and biological information measured in theuser while associating these pieces of information with one another. Theprocessor 26 converts the biological information regarding the user intoemotion index data indicating an emotion, stress, concentration,relaxation, burden, understanding, proficiency, interest, or the like.

Electrodermal activity, for example, may be used as a parameterindicating a mental activity state. A skin potential level (SPL), whichis a direct current component of skin potential activity (SPA), may beused, instead. The SPL usually shows a large negative value when theuser is highly aroused (i.e., excited) and a large positive value whenthe user is sleepy or relaxed. Skin potential response (SPR), which isan alternating current component of the SPA, may be used, instead. TheSPR frequently occurs due to stimuli based on changes in an externalenvironment, such as a pain, a touch, a sound, and an image, deepbreathing, body movement, a thought, and the like.

The processor 26 displays information indicating the state of the userobtained from an interface such as a camera or a keyboard, a degree ofthe state indicated by the emotion index data (e.g., an emotion, stress,concentration, relaxation, burden, understanding, proficiency, interest,etc.), and a type of state. For example, the processor 26 changesdisplay color, a type of font of letters, and the size of letters inaccordance with the degree of the state and the type of state. Forexample, the processor 26 may change a speech style (e.g., pleasure,anger, sadness, fear, kindness, etc.) or a tone of voice (e.g.,emphasis, speed, pitch, intonation, volume, etc.) in accordance with astimulus response (e.g., relaxation, stress, tension, etc.) in a readingfunction achieved by known software. The processor 26 may change thevolume or speed of voice of the reading function in accordance with thedegree of the user's emotion.

The above examples may be applied, for example, to an electronic book,movie content (e.g., includes content delivered on the Internet), anelectronic learning material, a tool for an online meeting, or the like.

In yet another example, when a document or a material is created using afunction of changing the size or color of letters and figures inaccordance with biological information regarding the user, the documentor the material can lack consistency if the function is used all thetime. The processor 26, therefore, need not change the size or color ofletters and figures in accordance with the biological information incertain areas of the document or the material. In other words, theprocessor 26 may change, in accordance with the biological informationregarding the user, the size or color of letters and figures in areaswhere such changes are permitted.

If words written in the document or the material are read aloud usingthe reading function, the processor 26 may change the volume and speedof the reading in accordance with the biological information regardingthe user. The speed of the reading may be limited to a certain range sothat the reading does not become too fast or too slow.

Time taken to complete a process for determining an emotion, a mentalstate, or a psychological state on the basis of biological informationand time taken to complete measurement of an emotion, a mental state, ora psychological state can vary depending on a type of emotion, mentalstate, or psychological state. Time intervals of measurement of a degreeof concentration or relaxation, for example, are 1 second, timeintervals of measurement of a degree of interest, understanding,proficiency, an activity load, an emotion, or creativity are 1 to 5seconds, and time intervals of measurement of comfort or discomfort are5 or 10 seconds. These time intervals are just examples, and may changedepending on the performance of the biological information measuringapparatus 12 or the like.

When the time intervals vary as described above, the processor 26 neednot determine all of predetermined emotions, mental states, andpsychological states and may determine only some predetermined emotions,mental states, and psychological states or limit time periods or scenesin which the determination is made. When the user is studying, forexample, the processor 26 may determine only an effect of the study onthe basis of the biological information regarding the user and need notdetermine other emotions. If the user is in a certain state (e.g., ifthe user has a certain disease such as an underlying disease), however,the processor 26 may determine all of the predetermined emotions, mentalstates, and psychological states all the time.

If the user has taken off the hearable but is wearing another biologicalinformation measuring apparatus 12, the processor 26 may determine anemotion, a mental state, or a psychological state on the basis ofbiological information measured by the other biological informationmeasuring apparatus 12. If the user was wearing a hearable and asmartwatch and has taken off the hearable, for example, an emotion, amental state, or a psychological state of the user may be determined onthe basis of biological information measured by the smartwatch.

Nineteenth Example

A nineteenth example will be described hereinafter. If biologicalinformation in a certain frequency band (e.g., brain waves in a certainfrequency band) is measured in the above examples and biologicalinformation in a frequency band other than the certain frequency band isnot measured, the processor 26 may estimate the biological informationin the other frequency band. For example, the processor 26 estimatesbiological information that is of the same type as the measuredbiological information but that is in a frequency band other than afrequency band including the measured biological information.

If only a potential in a frequency band of alpha waves, which are anexample of brain waves, is measured and a potential in a frequency bandother than that of alpha waves is not measured, for example, theprocessor 26 estimates brain waves (e.g., brain waves other than alphawaves, such as beta waves) in the other frequency band that have notbeen actually measured and brain waves in the frequency band in whichbrain waves have not been clearly detected on the basis informationmeasured by various sensors (e.g., a gyro sensor, an accelerationsensor, etc.), environment information (e.g., information indicatingtemperature, humidity, weather, a position, etc.), information regardingactivity of the user (e.g., information indicating exercise, desk work,sleep, etc.), information indicating a disease of the user, and/orinformation indicating a history of measurement of the measured alphawaves (e.g., information indicating waveforms, periods, and amplitude ofthe alpha waves) at the time of the measurement. For example, theprocessor 26 estimates, on the basis of correlation between the abovevarious pieces of information, brain waves in the frequency band thathave not been measured. The processor 26 may output the estimated brainwaves in the other frequency band to a terminal apparatus or the like.For example, the estimated brain waves in the other frequency band aredisplayed on a display of the terminal apparatus. At this time, themeasured brain waves (e.g., alpha waves) in the frequency band and theestimated brain waves (e.g., beta waves) in the other frequency band maybe displayed as graphs having the same scale (e.g., graphs whosehorizontal axes represent time and vertical axes represent potential).The processor 26 may output the estimated brain waves in the otherfrequency band as candidates for brain waves in the frequency band. Whenalpha waves have not been measured and only beta waves have beenmeasured, too, the alpha waves may be estimated.

When biological information in plural frequency bands have beenmeasured, the processor 26 may estimate, on the basis of the biologicalinformation in the frequency bands, biological information in anotherfrequency band that has not been measured. When alpha waves and betawaves have been measured and other brain waves have not been measured,for example, the processor 26 may estimate, on the basis of the measuredalpha waves and beta waves, brain waves in a frequency band other thanones including the alpha waves and the beta waves.

When plural candidates have been estimated, the processor 26 may displaythe candidates on the display of the terminal apparatus or output asound indicating that the candidates have been estimated. The processor26 may change order in which the candidates are displayed or displayscores on the basis of probabilities or possibilities of occurrence ofthe candidates. When a hearable measures brain waves, for example, itmight be difficult to detect biopotentials caused at positions on thehead relatively far from the ears. In this case, brain waves in afrequency band that have not been measured can be estimated on the basisof brain waves in another frequency band that have been measured oranother piece of information (e.g., environment information orinformation regarding the user).

Whether to enable or disable a function of estimating biologicalinformation in a frequency band that has not been measured may be set bythe user. If the user enables the function using the terminal apparatusor the like, for example, the function is executed. If the user disablesthe function using the terminal apparatus or the like, the function isnot executed. Because biological information that has not been measured(e.g., brain waves in a frequency band that have not been measured)might not be actually caused in the user's brain, the function can bedisabled in such a case.

Twentieth Example

A twentieth example will be described hereinafter.

When brain waves or the like are used for communication between personsas a BMI, information to be communicated is identified, and means forcommunicating the information is provided.

For example, a communication target is extracted and information to becommunicated is identified. A person who transmits the information thennotifies a person who receives the information of the information. Onlyone communication target (a person who receives information) may beprovided for a single person who transmits information, or pluralcommunication targets (e.g., multiple communication targets) may beprovided for a single person who transmits information. The informationto be communicated is information regarding a thought or an intention ofa person or state information and may be, for example, approval,agreement, disagreement, rejection, hesitation, confusion, etc. Wheninformation is provided for a person, the person transmits his/herthought about the information, namely whether he/she approves theinformation, to a person who receives the information. A method fortransmitting information to a person is a method for giving a sensorystimulus, such as transmission of information by sound, transmission ofinformation by light, or transmission of information by vibration. Morespecifically, in the case of a hearable, for example, measurement ofbiological information such as brain waves (measurement of informationto be communicated) and transmission of information by sound can beachieved simultaneously using the ears, that is, the BMI can be achievedwithout preparing various devices. The sound may be languageinformation, but may be musical tones, chimes, or the like, instead,insofar as what the musical tones, the chimes, or the like mean isdetermined in advance. Two successive sounds of a chime may meanapproval, for example, or different levels or degrees of an intentionmay be transmitted by changing pitch. A low chime may mean a low levelof approval, and a high chime may mean a high level of approval. Withthe advent of such a BMI, communication between persons can be achievedby various methods based on biological information such as brain waves,in addition to conventional methods based on language that employ soundor a graphical user interface (GUI).

Specific examples of a process in the twentieth example will bedescribed hereinafter with reference to FIGS. 27 and 28. FIGS. 27 and 28are flowchart illustrating the specific examples of the process. FIG. 27illustrates a process at a time when a device is operated. FIG. 28illustrates a process at a time when persons communicate with eachother.

As illustrated in FIG. 27, first, an operation mode is selected (S01),and operation intention information regarding a user is extracted anddetected (S02). As a result, a device to be operated and an operationare extracted and associated with each other. A method for determininglikelihood of an operation intention is presented to the user. If thelikelihood of the operation intention is high (HIGH in S03), theoperation is immediately performed on the device (S04). Correctness ofthe operation is then verified (S05). For example, the user performs theverification. If the operation is correct (YES in S06), the device willbe operated in the same manner next time (S07). For example, theoperation is set for the device, and the same operation will beperformed next time. If the operation is not correct (NO in S06), theprocess proceeds to step S11. If the likelihood of the operationintention is low (LOW in S03), the operation intention is checked byanother method (S08). If the likelihood of the operation intention ishigh, too, the operation intention may be checked by another method.Information indicating another method, for example, is displayed on adisplay or output from earphones of a hearable as a sound. As a result,the other method is presented to the user. If the checked operationintention is corrected (YES in S09), that is, if the checked operationintention matches an operation intention of the user, the device isoperated (S10). If the checked operation intention is not correct (NO inS09), the process proceeds to step S11. In step S11, a reason why theoperation intention is not correct is analyzed. For example, the userconducts the analysis. A next determination method (i.e., a method fordetermining an operation) is selected on the basis of a result of theanalysis (S12). A criterion for determining an operation intention ischanged (S13), and the processing in step S03 and the later stepsreflects a new criterion. For example, a threshold registered in themanagement table is changed, and an operation intention is checked orthe likelihood is determined, for example, on the basis of a newthreshold. The user may perform the selection in the above process.

When persons communicate with each other, a communication mode isselected as illustrated in FIG. 28 (S20). The user sets an automaticselection mode or a manual selection mode, for example, and thecommunication mode is selected in accordance with the setting. Next, acommunication target is selected (S21). In a mode in which thecommunication target is automatically selected, for example, an attendeeof a meeting or the like is selected as the communication target. In amode in which the user manually selects the communication target, forexample, candidates for the communication targets are presented to theuser using a sound output from the hearable or information displayed ona display, and the user selects the communication target. Next, the userselects information that may be transmitted to the communication target(S22). For example, the user selects the information that may betransmitted to the communication target among pieces of intentioninformation and pieces of state information. In addition, the user mayselect information that the user does not desire to transmit to thecommunication target. Information indicating an emotion, for example, isselected as the information that the user does not desire to transmit tothe communication target. Next, biological information is measured orobserved, and intention information, state information, and the like areextracted on the basis of the biological information (S23). Next,information to be transmitted to the communication target is detectedfrom the extracted intention information and state information and thelike (S24). For example, a detection test is conducted in advance, andintention information, state information, or the like similar toinformation detected in the detection test (e.g., intention information,state information, or the like whose degree of similarity is higher thanor equal to a threshold) is detected from the extracted intentioninformation and state information and the like. Methods for detectingsuch information may be presented to the user. The detected intentioninformation, state information, or the like is then transmitted to thecommunication target. For example, methods for transmitting suchinformation may be presented to the user, and the user may select one ofthe methods. The methods include, for example, transmission by vibration(sound) and transmission by light. Information may be transmitted to thecommunication target using language, music, luminous signals, or thelike. After the information is transmitted to the communication target,the user, who is a sender of the information, may be notified ofcompletion of the transmission. The notification may be omitted,instead. After step S24 is performed, the processing in step S203 andthe later steps illustrated in FIG. 27 may be performed.

The process in the twentieth example may be performed using a hearable.Because biological information is measured with the hearable worn on theuser's ears, the user can be notified of various pieces of informationwhile the biological information is being measured. Since the hearableis worn on the user's ears, the hearable has a natural appearancecompared to an electroencephalogram worn on the head in order to measurebrain waves or the like. The likelihood in the operation mode and thecommunication mode increases when a BCI based on the hearable is used.

In the above examples, a part or the entirety of the processes performedby the processor 26 of the information processing apparatus 10 may beperformed by the processor 34 of the terminal apparatus 18 or thebiological information measuring apparatus 12, instead. Alternatively, apart or the entirety of the processes may be performed by one or moreapparatuses selected from the information processing apparatus 10, theterminal apparatus 18, and the biological information measuringapparatus 12.

Twenty-first Example

It is known that a person's state of mind (e.g. a decision) in regardsto performing some task or solving a problem is reflected in theperson's brain waves. For instance, when a person gives up performingsome task, a theta wave increases, while when a person successfullysolves a problem, an alpha wave increases(http://qlifepro.com/news/20211124/eeg-rhythm.html).

In the twenty-first example, the biological information measuringapparatus 12 measures the brain waves of the user, and the informationprocessing apparatus 10 determines the situation that the user is inbased on the measured brain waves and determine a type of interventionthat may be needed by the user. Depending on the determination, anintervention device (e.g. the terminal apparatus 18) may perform theintervention by sending direct stimulation to the user (such as byelectric stimulation or playing music) to affect the user's brain wavestate so as to affect the user's state of mind, or to alert a thirdparty (such as teachers, parents, user's superior) of the state of theuser to provide appropriate help/intervention to the user.

More generally, the biological information measuring apparatus 12acquires biological information of at least one user. Then, theinformation processing apparatus 10 determines a state of at least oneuser based on the biological information including at least brain wavesof the at least one user, and historical information including at leastone time series of context information indicating contexts under whichmeasurements of the biological information at respective points time aremade. Then, the state of the at least one user may be output to the atleast one user.

The historical information may include at least one time series oraction history of the user or time series of context informationindicating contexts under which measurements of the biologicalinformation at respective points in time are made. For example, thecontext information may include: who makes the measurement, when themeasurement is made, where the measurement is made, what is measured,why it is measured, how it is measured, what was the user doing beforemeasurement is made, what was the user doing with other users beforemeasurement is made, the environmental information such as the position,location, building, room, or temperature of the environment in which theuser makes the measurement, scene or event that the user isparticipating in, or any of the other information indicating the contextunder which the measurement is made. Moreover, the historicalinformation may be searchable (by the user or by other automaticprocesses such as AI) in terms of the type of contents it contains (e.g.searchable in terms of events or scene, for instance) and/or in terms ofdifferent time ranges. The historical information may be obtained by acamera, a microphone, or from information recorded on the internet.

The historical information may be classified according to pluraldifferent time series each having different interval of time betweenwhich the successive biological or context information are collected ordifferent number of time points at which the biological or contextinformation are collected, and at least one determination regarding thestate of the at least one user is output based on analyzing thebiological information and the historical information for each of thetime series. For example, the state of the at least one user may bedetermined based on a combination of the biological information and thehistorical information collected at a first time series having timepoints that are apart by a first time interval (e.g., short timeinterval) and during a second time series having time points that areapart by a second time interval (e.g., long time interval).

Furthermore, a confidence for the determined state of the at least oneuser may be evaluated based on the similarity or sameness of thedetermined state for different time series. For instance, a higherconfidence is assigned if the results from the short time interval andthe long time interval are the same compared to if they are different.The determined state may be output to the user or be ignored based onthe determined confidence.

Each time series may also be assigned a weight by the user depending onthe context that the user is currently in and the determined states maybe output to the user in the order of priority determined by the weightsor a subset of the determined states may be output to the user dependingon the weights.

In other words, whether to output which result from which time seriesmay be determined based on the context information so that the resultthat best matches the user's expectation for a given situationdetermined by the context may be output. The determination that bestmatches the user's expectation may be set for various contexts by theuser or may be learned using an AI based on past determination.

For example, the determined state from the short time interval mayindicate abnormality, while the determined state from the long timeinterval may indicate normality. If the context indicates that the shorttime abnormality may be ignored, such as the case when the determinedstate is erroneous or is outlier information (e.g. when there is acontact failure of the device or when external but short and temporarystress is put on the user), it may be determined not to output theresults to the user. In other words, in such cases, the results may beoutput to the user only when the determined results for the short timeinterval and the long time interval are the same. On the other hand,context may indicate that the short time abnormality may be important,such as when the user is sick and any short-time change in the user'sstate needs to be reported. In such a case, the result from the shorttime interval may be assigned higher weight and be output with higherpriority.

Furthermore, the length of the time intervals or number of time pointsmay change or set in accordance with the context (a scene or event) thatthe user is in when the measurements are made. This correspondencebetween the context to the length of the time intervals or the number ofpoints may be set by the user or may be learned by an AI based on thepast data.

Note, that some examples above are illustrated with respect to two timeseries with two time intervals (short or long), but the presentembodiment is not limited, and is applicable to any number of timeseries.

Alternatively, a suggested intervention may be determined based on thestate of the at least one user and/or the historical information, andthe suggested intervention may be output to the at least one user.

Alternatively, the biological information measuring apparatus 12 may bein the form of an audible device (as discussed in below) including asound input-output device, and the information processing apparatus 10may output the suggested intervention to the at least one user via thesound input-output device, acquire a response of the at least one userto the suggested intervention, and store the response in the memory. Thesuggested intervention and the response of the at least one user may bestored as part of the historical information. In this way, determinationof the suggested intervention may take into account past interventions,responses and their context when the user was in a similar or a samestate or context, as well as the context that the user is currently in.The correspondence between the context to the suggested intervention maybe set by the user or may be learned by an AI based on the past data.

Alternatively, before performing the suggested intervention, theinformation processing apparatus 10 may select a first mode in which theresponse is acquired from the at least one user, or a second mode inwhich the suggested intervention is automatically performed byartificial intelligence.

Alternatively, the intervention may be performed by an interventiondevice (e.g. a terminal apparatus 18) in a form or mode of communicatingvia at least one of light (by way of light color, light intensity, orlight pulses), image, video, sound, voice, music, vibration, temperaturechange (hot or cold), impact (collision with an object, or throwing anobject), stimulation (pinching or pulling), or smell/scent. An examplein which the intervention is communicated by light is illustrated athttps://www.youtube.com/watch?v=Vr8hSeAjY2I.

Other forms or modes of intervention may include an adjustment of airtemperature (e.g. temperature setting on an air conditioner), aninteraction with an AI (e.g. comforting, encouraging, helping or makingan advice), or a comforting expression or gesture by a robot (e.g. asound coming from a pet robot or behaviors/actions of the pet robot thatare comforting).

Alternatively, the state of the at least one user may be output to atleast one other user.

In the case of outputting to the at least one other user, thedetermination of the state of the at least one user may be performed bythe information processing apparatus 10 of the at least one other userbased on the biological information and the historical information ofthe at least one user or the at least one other user. Similarly, thesuggested intervention may be performed by the intervention deviceshared by the at least one user and the at least one other user.Alternatively, the biological information of the at least one user maybe transmitted to the at least one other user and the informationprocessing apparatus 10 of the at least one other user may determine theintervention (or the intervention device) to be used.

When the at least one user and the at least one other user is in thesame location, the same intervention device may be selected forperforming the intervention. More specifically, the range of spacewithin which the state of the at least one user is shared or theintervention device to be used can be determined based on locationinformation of the at least one user and the at least one other user.Note that such sharing of the state of the at least one user can bedetermined by the type of relationship between the at least one user andthe at least one other user (e.g. a relationship between a patient and adoctor, or a relationship between workers in a cooperativerelationship).

The determination of the state of the at least one user or thedetermination of the intervention may be based on the past determinationof the at least one user and the at least one other user, which addsvariability to the determination done solely by relying on informationof one user. Moreover, the biological information of the at least oneother user may be found to be similar to or same as the biologicalinformation of the at least one user, in which case the at least oneother user may be added as a candidate to whom the same intervention canbe performed. In determining the intervention itself or selecting whichintervention device to use, past interventions (or intervention devices)in terms of their effectiveness may be recorded in the historicalinformation and a priority may be assigned such that the intervention(or intervention device) that had the highest effect (that has a higherpriority) may be selected. Learning of which intervention (orintervention device) is effective based on past data can be done by anAI. In the case of there being a new candidate for intervention (orintervention device) or if there are plural candidates for intervention(or intervention device), the user may select from among the candidates.Alternatively, the at least one other user may select the candidateintervention (or intervention device) based on its past effectiveness onthe at least one user. Alternatively, the at least one user may decidenot to select the intervention (or intervention device) but permit theat least one other user to whom a permission has been given to selectthe intervention (or intervention device) on behalf of the at least oneuser, for there are cases in which selection of an intervention byothers may be more effective than by oneself.

Twenty-Second Example

The information processing apparatus 10 may also be used to assistcommunication of at least one user.

In the twenty-second example, the biological information measuringapparatus 10 acquires biological information of at least one user. Theinformation processing apparatus 10 determines a state of at least oneuser based on the biological information of the at least one user. Then,the information processing apparatus 10 assists communication ofinformation of the at least one user based on the state of the at leastone user.

Alternatively, when assisting communication of information from one userto other user, the information processing apparatus may obtainconfirmation from the one user to transmit the information to the otheruser.

Alternatively, when the one user rejects the information to betransmitted to the other user, the information may be stored in thememory associated with the one user.

Alternatively, when the one user rejects the information to betransmitted to the other user, a selection of at least one alternateaction may be suggested to the one user. The alternate action mayinclude at least one of retransmitting the information to the other userafter a predetermined time period passes, transmitting the informationto a different user, or abandoning transmission of the information.

Alternatively, when the one user confirms transmission of theinformation to the other user, the information processing apparatus 10may store, as learned data, the information in association with thestate of the one user in the memory, and use the learned data insubsequent determination of the information based on the state of theone user.

Alternatively, when the one user confirms transmission of theinformation to the other user, the information processing device maystore data indicating whether or not the transmission to the other userwas successful.

Alternatively, the information processing apparatus 10 may determine thestate of the at least one user based on frequency spectrum of abioelectrical potential measured from the at least one user.

Hearable

A hearable, which is an example of the biological information measuringapparatus 12 according to the present exemplary embodiment, will bedescribed in detail with reference to FIGS. 29 to 37. Although earphonesto be worn on a person's ears will be taken as an example of thehearable hereinafter, the hearable to be used need not be earphones.

For example, at least one electrode is used as a biopotential detectionelectrode (hereinafter referred to as a “sensor electrode”) fordetecting a potential including biopotential, and at least one electrodeis used as an electrode for grounding (hereinafter referred to as a“ground electrode”). At least one electrode is used as an electrode(hereinafter referred to as a “reference electrode”) for detecting apotential to be compared with the potential detected by the sensorelectrode.

A potential detected by the sensor electrode will be referred to as a“sensor potential”, a potential detected by the reference electrode willbe referred to as a “reference potential”, and a potential detected bythe ground electrode will be referred to as a “ground electrode”hereinafter.

FIG. 29 is a perspective view of the entirety of a hearable 200. FIG. 30is a diagram illustrating the hearable 200 viewed from above. FIG. 31 isa diagram illustrating a left earphone viewed from the left. FIG. 32 isa perspective view of a right earphone. FIG. 33 is a perspective view ofthe left earphone. FIG. 34 is a diagram illustrating the right earphoneviewed from above. FIG. 35 is an exploded perspective view of the rightearphone. FIG. 36 is a cross-sectional view of the right earphone. FIG.37 is a perspective view illustrating the inside of the left earphone.

As illustrated in FIG. 29, front and back directions, up and downdirections, and left and right directions are defined for convenience ofdescription. The front direction is a direction in which a person's faceis oriented, and the back direction is a direction opposite the frontdirection. The up direction is a direction in which a top of theperson's head is oriented, and the down direction is a directionopposite the up direction. The right direction is a direction of theperson's right hand, and the left direction is a direction of theperson's left hand. The front and back directions, the up and downdirections, and the left and right directions are perpendicular to oneanother.

It is assumed here, for example, that the hearable 200 measuresbiological information including brain waves. The hearable 200 maymeasure biological information other than brain waves in addition to, orinstead of the brain waves. When a right earphone 216R and a leftearphone 216L, which will be described later, are worn on a person'sears, for example, a potential indicating biological information isdetected. A potential signal indicating the detected potential usuallyincludes a potential signal indicating brain waves. In addition to thepotential signal indicating brain waves, however, the potential signalmight include a potential signal indicating a potential caused bymovement based on a facial expression or a potential caused by movementof the neck, the jaw, or the eyes as a potential signal indicating apotential caused by movement of the person's head. In addition, cerebralblood flow, pulse waves relating to cardiac blood flow, heartbeat, orthe like might be clearly measured as biological information caused bychanges in blood flow due to movement of the person's head. The hearable200 might thus measure biological information other than brain wavesalong with the brain waves. The biological information other than brainwaves may be processed as noise (e.g., may be removed), or separatedfrom the brain waves and used for some processes.

As illustrated in FIGS. 29 and 30, the hearable 200 roughly includes theright earphone 216R worn on the right ear, the left earphone 216L wornon the left ear, and a cable 218 for electrically connecting the rightearphone 216R and the left earphone 216L to each other. The rightearphone 216R and the left earphone 216L may communicate signals anddata with each other through the cable 218. The cable 218 may beprovided with a remote control for controlling the hearable 200.

Either the right earphone 216R or the left earphone 216L may function asa biological information measuring apparatus that measures biologicalinformation, or both the right earphone 216R and the left earphone 216Lmay function as biological information measuring apparatuses.

For example, one of the earphones may include the sensor electrode, thereference electrode, and the ground electrode, and the earphone need notinclude any electrode.

In another example, both the right earphone 216R and the left earphone216L may include the sensor electrode, the reference electrode, and theground electrode.

In yet another example, one of the earphones may include one or two ofthe sensor electrode, the reference electrode, and the ground electrode,and the other earphone may include an electrode that is not included inthe foregoing earphone.

In yet another example, the right earphone 216R and the left earphone216L may each include one or more electrodes, and the sensor electrode,the reference electrode, and the ground electrode may be distributedbetween the right earphone 216R and the left earphone 216L.

In yet another example, plural sensor electrodes, plural referenceelectrodes, and plural ground electrodes may be provided for one of theearphones. For example, plural sensor electrodes may be provided for theright earphone 216R, and plural reference electrodes may be provided forthe left earphone 216L.

The right earphone 216R is a canal-type earphone, for example, andincludes a right case 220R, a right earpiece 222R, a right support 224R,and a right ear hook 226R, and an electrode member 228R. The earpiece issometimes called an “earpad”.

The right case 220R has a thin rectangular parallelepiped shape, forexample, and stores an electronic circuit and the like. The rightearpiece 222R and the right support 224R are arranged in the right case220R on a surface that faces the user's right ear when the user wearsthe hearable 200. The right case 220R stores a control device, aspeaker, a communication device (e.g., a communication chip), anelectronic circuit that analyzes and processes biological information, asix-axis sensor (e.g., a sensor including a three-axis sensor thatdetects acceleration and a three-axis sensor that detects angularvelocity etc.), a memory, a GPS sensor, and the like. The communicationdevice is a device for achieving a wireless communication function suchas Bluetooth (registered trademark) or BLE. The communication device mayinclude a wireless LAN (e.g., a network that employs Wi-Fi (registeredtrademark) etc.) or a cellular (3G, 4G, 5G, low-power wide-area (LPWA),etc.) module to achieve a wide communication area and directly transmitdata to remote devices through local area devices whose communicationdistances are longer than that of Bluetooth (registered trademark) orthe Internet. The six-axis sensor detects a traveling direction,orientation, and rotation of the right case 220R. Biological informationmay be stored in the memory. The electronic circuit that analyzesbiological information need not be provided for the biologicalinformation measuring apparatus 12.

The right support 224R has a columnar shape such as a cylindrical shapeand protrudes from a side (e.g., a surface that faces the user's rightear when the user wears the right earphone 216R on his/her right ear) ofthe right case 220R between the right case 220R and the right earpiece222R. For example, the outer diameter of the right support 224R islarger than that of the right earpiece 222R. The electrode member 228Ris provided for a part or the entirety of a side of the right support224R.

The electrode member 228R has a ring shape, for example, and issupported by the columnar right support 224R. A part or the entirety ofthe electrode member 228R functions as an electrode. That is, anelectrode may be provided all over a surface of the electrode member228R, or an electrode may be provided over a part of the surface of theelectrode member 228R and need not be provided over the rest of thesurface. The electrode member 228R is composed, for example, of aconductive rubber made of carbon. In this case, too, the entirety (e.g.,the entirety of the surface) of the electrode member 228R may becomposed of the conductive rubber and function as an electrode, or apart (e.g., a part of the surface) of the electrode member 228R may becomposed of the conductive rubber and function as an electrode.

The right earpiece 222R is provided at a tip of the right support 224R(i.e., an end of the right support 224R opposite an end connected to theright case 220R). For example, the right earpiece 222R has a cylindricalshape that tapers toward a tip. This shape is just an example, and theright earpiece 222R may have another shape, instead.

The right earpiece 222R stores a sound conduit (e.g., a conductive tubemember that will be described later) and the like. A sound output fromthe speaker goes out of the right earpiece 222R through the soundconduit and the like inside the right earpiece 222R. An electrode isprovided over a part or the entirety of an outer surface (e.g., a sideetc.) of the right earpiece 222R. The electrode is composed, forexample, of conductive rubber made of carbon. The right earpiece 222Ritself may be composed of the conductive rubber, instead. For example, apart or the entirety of the right earpiece 222R may be composed of theconductive rubber. That is, a part or the entirety of the surface of theright earpiece 222R may function as an electrode.

The right earpiece 222R and the electrode member 228R may be, forexample, elastic members. A resin such as rubber is used for the elasticmembers. More specifically, a silicone-based rubber (e.g., 51734manufactured by NOK Corporation), a urethane-based rubber, or the likemay be used for the right earpiece 222R and the electrode member 228R.The hardness of the right earpiece 222R and the electrode member 228R(e.g., hardness according to specifications of a type A durometer(instantaneous)) is, for example, 40 to 75. A resin having a hardness of70, for example, may be used for the right earpiece 222R and theelectrode member 228R.

As described later, the right earpiece 222R is arranged on the right earin such a way as to be inserted into the right ear canal and come intocontact with the right ear canal, and the electrode member 228Rsupported by the right support 224R is arranged on the right ear in sucha way as to come into contact with the right ear cavity.

The right ear hook 226R has a curved shape as a whole and is hung on theuser's right ear when the user wears the right earphone 216R. Forexample, the right ear hook 226R is hung on the right ear helix. Morespecifically, the right ear hook 226R is arranged behind the right earhelix while in contact with the right ear helix. An end of the right earhook 226R is connected to a front part of the right case 220R. The rightear hook 226R bends from the connection with the right case 220R towarda back part of the right case 220R, thereby forming a curve. The curveis arranged behind the right ear helix while in contact with the rightear helix. For example, the curve has a shape along a shape of the backof the right ear helix. Another end of the right ear hook 226R isconnected to an end of the cable 218.

The right earpiece 222R and the right support 224R are replaceablemembers. For example, plural (e.g., three to five each) right earpieces222R and right supports 224R having different shapes and sizes areprepared. The right earpiece 222R and the right support 224R may bereplaced in accordance with the shape of the user's right ear (e.g., theear canal, the ear cavity, or another part).

The left earphone 216L is a canal-type earphone, for example, andincludes a left case 220L, a left earpiece 222L, a left support 224L, aleft ear hook 226L, and an electrode member 228L.

The left case 220L has a thin rectangular parallelepiped shape, forexample, and stores a battery and the like. The left earpiece 222L andthe left support 224L are arranged in the left case 220L on a surfacethat faces the user's left ear when the user wears the hearable 200.Power from the battery is supplied to the right earphone 216R and theleft earphone 216L to drive the right earphone 216R and the leftearphone 216L. For example, power from the battery is supplied tospeakers and circuits. The battery may be provided for both the rightcase 220R and the left case 220L or either the right case 220R or theleft case 220L.

The left support 224L has a columnar shape such as a cylindrical shapeand protrudes from a side (e.g., a surface that faces the user's leftear when the user wears the left earphone 216L on his/her left ear) ofthe left case 220L between the left case 220L and the left earpiece222L. For example, the outer diameter of the left support 224L is largerthan that of the left earpiece 222L. The electrode member 228L isprovided for a part or the entirety of a side of the left support 224L.

The electrode member 228L has a ring shape, for example, and issupported by the columnar left support 224L. A part or the entirety ofthe electrode member 228L functions as an electrode. That is, anelectrode may be provided all over a surface of the electrode member228L, or an electrode may be provided over a part of the surface of theelectrode member 228L and need not be provided over the rest of thesurface. The electrode member 228L is composed, for example, of aconductive rubber made of carbon. In this case, too, the entirety (e.g.,the entirety of the surface) of the electrode member 228L may becomposed of the conductive rubber and function as an electrode, or apart (e.g., a part of the surface) of the electrode member 228L may becomposed of the conductive rubber and function as an electrode.

The left earpiece 222L is provided at a tip of the left support 224L(i.e., an end of the left support 224L opposite an end connected to theleft case 220L). For example, the left earpiece 222L has a cylindricalshape that tapers toward a tip. This shape is just an example, and theleft earpiece 222L may have another shape, instead.

The left earpiece 222L stores a sound conduit and the like. A soundoutput from the speaker goes out of the left earpiece 222L through thesound conduit and the like inside the left earpiece 222L. An electrodeis provided over a part or the entirety of an outer surface (e.g., aside etc.) of the left earpiece 222L. The electrode is composed, forexample, of conductive rubber made of carbon. The left earpiece 222Litself may be composed of the conductive rubber, instead. For example, apart or the entirety of the left earpiece 222L may be composed of theconductive rubber. That is, a part or the entirety of the surface of theleft earpiece 222L may function as an electrode.

The left earpiece 222L and the electrode member 228L may be, forexample, elastic members. A resin such as rubber is used for the elasticmembers. More specifically, a silicone-based rubber (e.g., 51734manufactured by NOK Corporation), a urethane-based rubber, or the likemay be used for the left earpiece 222L and the electrode member 228L.The hardness of the left earpiece 222L and the electrode member 228L(e.g., hardness according to specifications of a type A durometer(instantaneous)) is, for example, 40 to 75. A resin having a hardness of70, for example, may be used for the left earpiece 222L and theelectrode member 228L.

As described later, the left earpiece 222L is arranged on the left earin such a way as to be inserted into the left ear canal and come intocontact with the left ear canal, and the electrode member 228L supportedby the left support 224L is arranged on the left ear in such a way as tocome into contact with the left ear cavity.

The left ear hook 226L has a curved shape as a whole and is hung on theuser's left ear when the user wears the left earphone 216L. For example,the left ear hook 226L is hung on the left ear helix. More specifically,the left ear hook 226L is arranged behind the left ear helix while incontact with the left ear helix. An end of the left ear hook 226L isconnected to a front part of the left case 220L. The left ear hook 226Lbends from the connection with the left case 220L toward a back part ofthe left case 220L, thereby forming a curve. The curve is arrangedbehind the left ear helix while in contact with the left ear helix. Forexample, the curve has a shape along a shape of the back of the earhelix. Another end of the left ear hook 226L is connected to another endof the cable 218.

The left earpiece 222L and the left support 224L are replaceablemembers. For example, plural (e.g., three to five each) left earpieces222L and left supports 224L having different shapes and sizes areprepared. The left earpiece 222L and the left support 224L may bereplaced in accordance with the shape of the user's left ear (e.g., theear canal, the ear cavity, or another part).

The control device, the communication device, the electronic circuit,the six-axis sensor, the memory, and the like may be stored in eitherthe right case 220R or the left case 220L or both the right case 220Rand the left case 220L.

The right case 220R or the left case 220L is provided with a powersupply button, a switch for adjusting sound volume, and the like. Thepower supply button, the switch, and the like may be provided for boththe right case 220R and the left case 220L.

For example, either the electrode provided for the right earpiece 222Ror the electrode provided for the left earpiece 222L is used as thesensor electrode, and the other electrode is used as the referenceelectrode. The electrode member 228R and the electrode member 228L areused as the ground electrodes. Alternatively, either the electrodemember 228R or the electrode member 228L may be used as the sensorelectrode, the other may be used as the reference electrode, and theelectrodes provided for the right earpiece 222R and the left earpiece222L may be used as the ground electrodes.

In another example, plural electrodes separate from one another may beprovided for the right earpiece 222R, and at least one of the electrodesmay be used as the sensor electrode, the reference electrode, or theground electrode. For example, the electrodes may be used as sensorelectrodes, and a potential detected by an electrode whose detectionsensitivity is the highest, an electrode whose noise is the smallest, oran electrode whose level of noise is the most stable may be used as asensor potential. The same holds when the electrodes are used asreference electrodes or ground electrodes. The same also holds for theleft earpiece 222L, the electrode member 228R, and the electrode member228L.

As illustrated in FIG. 34, the electrode member 228R supported by theright support 224R is arranged higher than the right case 220R by aheight h. That is, a distance between the electrode member 228R and theright case 220R is set at the height h. The height h is set, forexample, at a value with which the right case 220R and the ear helix donot interfere with each other and a contact failure does not occur. Morespecifically, the height h is set such that the right case 220R does notcome into contact with the right ear helix when the right earpiece 222Ris inserted into the right ear canal and the right earphone 216R is wornon the right ear. That is, the height h is set such that the right case220R becomes away from the right ear helix. In doing so, for example, anupper surface of the right case 220R does not come into contact with theright ear helix. If the right case 220R comes into contact andinterferes with the right ear helix, the right earpiece 222R is notinserted into the ear canal up to a position at which the right earpiece222R becomes stable, and a contact failure might occur between theelectrode of the right earpiece 222R and the ear canal. In addition, theelectrode member 228R might not be arranged at a position at which theelectrode member 228R has a good contact with the right ear cavity, anda contact failure might occur between the electrode member 228R and theright ear cavity. If such a contact failure occurs, detectionsensitivity of sensor potential, reference potential, or groundpotential might decrease, thereby decreasing measurement accuracy ofbiological information. By setting the height h such that the right case220R does not come into contact with the right ear helix, the right case220R and the right ear helix do not interfere with each other, the rightearpiece 222R can be inserted into the ear canal up to the position atwhich the right earpiece 222R becomes stable, and the electrode of theright earpiece 222R and the right ear canal have a good contact witheach other. In addition, the electrode member 228R can be arranged atthe position at which the electrode member 228R has a good contact withthe right ear cavity, and the electrode member 228R and the right earcavity have a good contact with each other. In doing so, detectionsensitivity of biopotential improves. The same holds for the leftearphone 216L.

As illustrated in FIG. 30, when the right earphone 216R is viewed fromabove, the right ear hook 226R inclines from the right case 220R towardthe right earpiece 222R. That is, when the right earphone 216R is viewedfrom above, there is a gap between the right case 220R and the right earhook 226R. In other words, the right ear hook 226R is offset from theright case 220R toward the right earpiece 222R. The gap is for the rightear, that is, the right ear is inserted into the gap. By providing sucha gap and make the user insert his/her right ear into the gap, theuser's right ear can be inserted into the gap regardless of the size ofthe right ear, so that the right earphone 216R stays on the right ear.As a result, the electrode of the right earpiece 222R and the right earcanal have a good contact with each other, and the electrode member 228Rand the right ear cavity have a good contact with each other. The sameholds for the left earphone 216L.

As illustrated in FIG. 35, a conductive tube member 230R composed of ametal is provided for the right support 224R and protrudes from theright case 220R. A bottom of the conductive tube member 230R is mountedon the right support 224R, and the right earpiece 222R is arranged at atop of the conductive tube member 230R. For example, the bottom of theconductive tube member 230R is screwed into a top of the right support224R. A conductive tube member 232R composed of a metal is provided fora side of the right support 224R and protrudes from the side. The ringelectrode member 228R is also provided for the side in such a way as tocover the conductive tube member 232R. More specifically, a groove isformed in the side of the right support 224R along a circumferentialdirection of the right support 224R, and a hole 224R1 into which an endof the conductive tube member 232R is to be inserted is formed in thegroove. When the end of the conductive tube member 232R is inserted intothe hole 224R1, the conductive tube member 232R is mounted on the sideof the right support 224R. For example, the conductive tube member 232Ris screwed into the side of the right support 224R. The electrode member228R is fitted into the groove in the side of the right support 224Rwith the conductive tube member 232R mounted on the side of the rightsupport 224R, and the electrode member 228R is mounted on the side ofthe right support 224R while in contact with the conductive tube member232R. The conductive tube member 232R is connected to electrical wire inthe right support 224R. As a result, a potential signal indicating apotential detected by the electrode member 228R is transmitted to theelectrical wire through the conductive tube member 232R, and then to aboard 234, which will be described later. The left earphone 216L has thesame configuration as the right earphone 216R.

As illustrated in FIG. 36, the right case 220R stores the board 234 anda sub-board 236. Boards other than the board 234 and the sub-board 236may also be stored in the left case 220L. As illustrated in FIG. 37, theleft case 220L stores a battery 238 and a relay board 240. The battery238 supplies power to the right earphone 216R and the left earphone 216Lthrough the relay board 240.

The cable 218 is hard enough to maintain an overall shape thereof. Forexample, the cable 218 extends from the right case 220R and the leftcase 220L rearward so that the cable 218 does not come into contact withthe back of the user's head or hair on the back of the user's head whenthe hearable 200 is worn on the user's ears. Since the cable 218 doesnot come into contact with the back of the user's head or the hair onthe back of the user's head, noise that would otherwise be caused bysuch a contact is not caused in biological information.

Even if the cable 218 comes into contact with the back of the user'shead or the like, a load due to the contact is transmitted to the cable218, and, as indicated by an arrow Y in FIG. 31, the entirety of theleft earphone 216L rotates rearward. As a result, the left earpiece 222Land the electrode member 228L are firmly fixed by the left ear. That is,the entirety of the left earphone 216L rotates in a direction in whichthe left earpiece 222L and the electrode member 228L are fixed morefirmly. As a result, the electrode of the left earpiece 222L and theleft ear canal have a better contact with each other and the electrodemember 228L and the left ear cavity have a better contact with eachother, thereby improving the detection sensitivity. The same holds forthe right earphone 216R.

The hearable 200 may be capable of achieving bone conduction. In thiscase, information can be transmitted to the user through boneconduction, not by sound. Even in the case of a deaf user, therefore,biological information regarding brain waves can be measured andinformation can be transmitted through bone conduction using thehearable 200.

The above-described examples may be implemented using the hearable 200,instead. That is, biological information regarding the user may bemeasured using the hearable 200, and the examples may be implementedusing the measured biological information. Plural types of biologicalinformation may be measured using the hearable 200 and a biologicalinformation measuring apparatus 12, and a state of a person (e.g.,emotion information, mental information, psychological information,etc.) may be obtained on the basis of the measured types of biologicalinformation, instead.

In the embodiments above, the term “processor” refers to hardware in abroad sense. Examples of the processor include general processors (e.g.,CPU: Central Processing Unit) and dedicated processors (e.g., GPU:Graphics Processing Unit, ASIC: Application Specific Integrated Circuit,FPGA: Field Programmable Gate Array, and programmable logic device).

In the embodiments above, the term “processor” is broad enough toencompass one processor or plural processors in collaboration which arelocated physically apart from each other but may work cooperatively. Theorder of operations of the processor is not limited to one described inthe embodiments above, and may be changed.

The foregoing description of the exemplary embodiments of the presentdisclosure has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thedisclosure to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the disclosure and its practical applications, therebyenabling others skilled in the art to understand the disclosure forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of thedisclosure be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing method comprising: by aprocessor: acquiring biological information of at least one user;determining a state of the at least one user based on biologicalinformation including at least brain waves of the at least one user, andhistorical information including at least one time series of contextinformation indicating contexts under which measurements of thebiological information at respective points in time are made; andoutputting the state of the at least one user to the at least one user.2. The information processing method according to claim 1, wherein thehistorical information is classified according to a plurality of timeseries, and at least one determined state of the at least one user isoutput based on analyzing the biological information and the historicalinformation for each of the time series.
 3. The information processingmethod according to claim 2, wherein each time series is assigned aweight depending on a context that the user is in and the at least onedetermined state of the at least one user is output in an order ofpriority determined by the weights or a subset of the at least onedetermined state of the at least one user is output to the userdepending on the weights.
 4. The information processing method accordingto claim 1, wherein the state of the at least one user is determinedbased on a combination of the biological information and the historicalinformation collected at a plurality of time series.
 5. The informationprocessing method according to claim 1, wherein the state of the atleast one user is output to at least one other user.
 6. The informationprocessing method according to claim 5, further comprising determining asuggested intervention based on the state or the historical informationof the at least one user or at least one other user, and outputting thesuggested intervention to the at least one user or the at least oneother user, and wherein at least one intervention device for outputtingthe suggested intervention is selected from an environment that the atleast one user is in.
 7. The information processing method according toclaim 1, further comprising determining a suggested intervention basedon the state or the historical information of the at least one user, andoutputting the suggested intervention to the at least one user.
 8. Theinformation processing method according to claim 1, further comprisingdetermining the state of the at least one user based on a frequencyspectrum of a bioelectrical potential measured from the at least oneuser.
 9. The information processing method according to claim 7,wherein: the processor is included in an audible device comprising abiological information measuring device, a sound input-output device,and a memory; and the method further comprising: by a processor,outputting the suggested intervention to the at least one user via thesound input-output device, acquiring a response of the at least one userto the suggested intervention, and storing the response in the memory.10. The information processing method according to claim 9, furthercomprising, before performing the suggested intervention, selecting afirst mode in which the response is acquired from the at least one user,or a second mode in which the suggested intervention is automaticallyperformed by artificial intelligence.
 11. The information processingmethod according to claim 7, further comprising performing interventionin a form of communicating via at least one of light, image, video,sound, voice, music, vibration, temperature change, collision,stimulation, or smell.
 12. An information processing method comprising:by a processor, acquiring biological information of at least one user;determining a state of the at least one user based on the biologicalinformation of the at least one user; and assisting communication ofinformation of the at least one user based on the state of the at leastone user.
 13. The information processing method according to claim 12,wherein, at least one user comprises a plurality of users, and, in acase of assisting communication of information from one user to anotheruser, the processor obtains confirmation from the one user to transmitthe information to the other user.
 14. The information processing methodaccording to claim 13, further comprising, in a case in which the oneuser rejects the information to be transmitted to the other user,storing the information in a memory associated with the one user. 15.The information processing apparatus according to claim 13, furthercomprising, in a case in which the one user rejects the information tobe transmitted to the other user, suggesting a selection from at leastone alternate action to the one user.
 16. The information processingapparatus according to claim 13, further comprising, in a case in whichthe one user confirms transmission of the information to the other user,storing, as learned data, the information in association with adetermined state of the one user in a memory, and using the learned datain subsequent determination of the information based on the determinedstate of the one user.
 17. The information processing method accordingto claim 13, further comprising, in a case in which the one userconfirms transmission of the information to the other user, storing in amemory data indicating whether or not the transmission to the other userwas successful.
 18. The information processing method according to claim12, further comprising determining the state of the at least one userbased on a frequency spectrum of a bioelectrical potential measured fromthe at least one user.
 19. An information processing apparatuscomprising: a processor configured to: acquire biological information ofat least one user; determine a state of at least one user based onbiological information including at least brain waves of the at leastone user, and historical information including at least one time seriesof context information indicating contexts under which measurements ofthe biological information at respective points in time are made; andoutput the state of the at least one user to the at least one user. 20.An information processing device comprising: a processor configured to:acquire biological information of at least one user; determine a stateof at least one user based on the biological information of the at leastone user; and assist communication of information among the at least oneuser based on the state of the at least one user.